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INTRODUCTION AND HIstTorY 
OF THE PROBLEM 


HIS STUDY was designed to investigate 
te validity of holistic projective tech- 
nique evaluations. The holistic method 
treats the tests and the clinician as an in- 
separable combination. The clinician’s ap- 
praisal is based on his total impression 
of the patient’s productions and this ap- 
praisal is the unit evaluated. The question 
of the particular signs or content employed 
by the clinician in forming his appraisal is 
beyond the boundaries of this kind of 
research. 

Of the previous research on holistic meth- 
ods, the matching studies are the most 
numerous (Krugman, 1942; Palmer, 1951; 
Waehner, 1942). In these, judges are asked 
to match personality sketches drawn from 
an analysis of some projective test with 
sketches obtained from some other source. 
Investigations of this kind are inadequate 
in a number of ways. Since general sketches 
are usually used it is not possible to tell in 
which areas the projective technique in 


1 This monograph is based upon a doctoral dis- 


sertation at New York University in which the 
research is reported in much greater detail (Silver- 
man, 1958). The study was carried out at the 
Psychiatric Clinic of the Court of Special Sessions 
of the City of New York. I wish to express my 
deepest appreciation to Jules S. Golden, director of 
the clinic, who graciously permitted me to conduct 
this research and without whose cooperation the 
procedures employed in this investigation would not 
have been possible. Thanks are also due to the ten 
staff psychiatrists who served as psychiatric judges. 
I also wish to express my gratitude to the follow- 
ing thirty clinical psychologists who evaluated the 
projective test material, kindly giving time and 
effort without compensation: Stanley Berger, Fred 
3rown, Renata Calabresi, Louis Feigenbaum, Flor- 
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question can evaluate well and in which 
areas poorly. A more crucial objection is 
that results become partially dependent on 
artifacts. Both the ability of the judges to 
match and the similarity in description of 
the reports to be matched become crucial 
determinants of the results. Spurious fac- 
tors such as these can be responsible for the 
wide range of correlations found for differ- 
ent studies of this kind. Moreover, as 
Cronbach (1949) points out, the matching 
technique does not indicate the degree of 
rightness and wrongness for each predic- 
tion. Thus, a particular match or mismatch 
may be determined by the smallest of 
coincidences. 

Some of the studies approaching projec- 
tive technique validation from a_ holistic 
point of view avoided the pitfalls of the 
matching method by restricting the psy- 
chologists’ task to designating a diagnostic 
category for the patient. A number of these 
have demonstrated a high degree of agree- 
ment between the psychologists’ diagnoses 
and the criterion (Benjamin & Ebaugh, 


1938; Chamber & Hamlin, 1957; Siegel, 


ence Halpern, Howard Halpern, Emanuel Hammer, 
Doris Heller, Robert Holt, Walter Kass, Gertrude 
Kurth, Frank Lachmann, Leah Levinger, Dorothy 
Litwin, Carola Mann, David Mann, Ruth Munroe, 
Martha Schon, Miriam Siegel, Herbert Spohn, 
Bernard Steinzor, Allen Williams, Berta Beller, 
Lawrence Epstein, Ladilly Harris, Leone Lesser, 
Adam Munz, Irving Schwartz, Irving Steingart, 
and Joan Trachtman. Acknowledgment is also due 
to Fred Brown, chairman, Robert Holt, and Isidor 
Chein, who as members of the doctoral committee 
gave guidance and encouragement throughout this 
study, and to Jacob Cohen who graciously offered 
me advice and assistance with respect to the statis- 
tical procedures employed 

The author is also Senior Psychologist at Mt. 
Sinai Hospital, New York City 
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1948). Investigations of this kind are im- 
portant inasmuch as they show in an ob- 
jective scientific manner the validity of pro- 
jective techniques in one particular area. 
However, the worth of projective tech- 
niques is purported to extend much further 
than merely placing an individual into a 
category of diagnosis. Clinical psychologists 
have long maintained that the richness of 
projective methods lies in their ability to 
discover the more refined and less obvious 
aspects of personality. Schafer (1954) has 
stressed the great value of the Rorschach in 
discerning defense mechanisms. Brown? be- 
lieves that much regarding personality 
dynamics and the person’s earliest percep- 
tions of significant figures can be elicited 
from projective material. Piotrowski® em- 
phasizes that self concept and role in life 
are frequently revealed in Rorschach re- 
sponses. 

A few attempts have been made to vali- 
date projective techniques in terms of the 
variables mentioned above. Typical of these 
is the famous one by Hertz and Rubinstein 
(1939). Here, a Rorschach was given to 
one § and interpreted “blindly” by three 
leading Rorschachers (Hertz, Beck, and 
Klopfer). The interpretations were vali- 
dated against clinical material gathered in 
14 interviews. From an over-all perusal of 
the four reports, a great deal of similarity 
was said to exist both among the Rorschach 
experts and between each of them and the 
validating criterion. Studies such as this, 
although dealing with more extensive mate- 
rial than just a diagnostic classification, 
leave much to be desired as validation at 
tempts. The fact that a single case is usually 
used is an obvious shortcoming. Even more 
important is the fact that the method used 
for evaluation does not allow for a system- 
atic comparison of the psychiatric and psy- 
chological evaluations. Moreover, in the 
absence of a numerical index, the amount 
of agreement between the psychologists 
and the criterion can only be subjectively 
judged by the individual reader. This task 


Personal communication, 1958 


Piotrowski, Z. 


Brown, F. 


Personal communication, 1955 


is sometimes made especially difficult by the 
fact that the psychotherapists and the pro- 
jective test evaluators discuss different as- 
pects of the same patient. 

Krugman (1942) tried to overcome these 
shortcomings by asking judges to rate on a 
four-point scale the degree of similarity be- 
tween Rorschach reports and case study 
abstracts for different areas of functioning. 
In 94% of the cases there was “essential” 
or “fair” agreement. Similarly, Symonds 
(1955) found average agreement at 65% 
when he himself used a scale to judge the 
degree of similarity between Rorschach re- 
ports written by seven experienced Ror- 
schachers and extensive case study material 
on a single patient. Although an improve- 
ment over the aforementioned studies, these 
still did not overcome all of the objections 
previously raised. The case study abstracts 
still contained material for areas the Ror- 
schach report did not deal with and vice 
versa ; and in terms of subjectivity in evalu- 
ating the degree of agreement, all that was 


accomplished was that the opinions of 


judges were substituted for the opinions of 


the reader. Hertz’s 1941 characterization of 
the case study method still aptly applied : 
“The case study approach, though quite 
fruitful, remains a clinical qualitative ap- 
proach rather than a quantitative scientific 
one. It may be that new statistical devices 
will be created that will be able to place this 
approach through a quantitative analysis of 
the clinical material itself in a position of 
good validation” (Hertz, 1941, p. 531). 
One device that has been so 
employed is the rating scale (Samuels, 1952; 
Saxe, 1950). A second quantitative tech- 
nique that has been used is the O sort. This 
is superior to usual rating scales since it 
asks the evaluator to judge items in.a rela- 


statistical 


tive sense (that is, the relevance of one item 
compared with others), while the rating 
scales demand that items be judged in abso- 
lute terms. The latter can well lead to 
spuriously low agreement since different 
raters, whose evaluations are to be corre- 
lated with each other, may adopt different 
anchoring points for their ratings. 

In light of this advantage of O sorts, it 
is surprising that only two studies could be 


found in the literature that used them in 
validating projective techniques. One was 
reported by Little and Shneidman (1955). 
This was a quantitative addendum to 
Shneidman’s (1951) Thematic Test Anal- 
ysis book which presented in report form 
appraisals by 17 “expert” psychologists of 
one patient's TAT and MAPS protocols. 
Statements were culled from these reports 
and items stating their converse were for- 
mulated. One hundred fifty of these items 
were chosen for quantitative assessment. 
These statements were resubmitted to the 
17 psychologists for QO sorting. The crite- 
rion measures were the sortings of the state- 
ments by 29 “competent clinicians” who in- 
dependently appraised the statements on the 
basis of a complete clinical record (therapy 
notes, hospital observations, etc.). Validity 
coefficients between each psychologist in the 
first group with every clinician in the second 
group ranged from —.07 to .70, with the 
mean at .45. 

The second study using QO sorts was re- 
ported by Fisher (1952). She compared the 
O sorts of eight “expert” clinical psychol- 
ogists with those of four psychiatrists for 
five patients, each of whom was receiving 
psychotherapy from one of the psychia- 
trists. Four of the psychologists based their 
evaluations on figure drawings of the Ss 
while the other four had accessible instead 
a Rorschach, a TAT, and a Stanford-Binet 
intelligence test. The over-all correlation 
between the first group of psychologists and 
the criterion was .195, while the correlation 
between the second group and the criterion 
was .365. This difference was ascribed to 
the greater number of the 
group had available. 

Despite their use of O sorts, these studies 


tests second 


were unsatisfactory in two major respects. 
First, in both studies only one O sort was 
utilized which included items on different 
levels of personality functioning. Thus, 
some of them were diagnostic statements, 
others referred to unconscious needs, still 
others were concerned with conscious feel- 
ings, and some even referred to overt be- 
havior. By having statements at so many 
different levels in the same O sort, spurious 
differences between evaluators were apt to 
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arise. One evaluator, because of personal 
preference, theoretical orientation, or the 
particular medium he works with, may give 
priority to statements on one level over 
those on another. Thus, differences between 
sorters were apt to arise due to artifacts 
rather than real disagreement. That this 
actually occurred is implicit in Little and 
Shneidman’s findings that statistical analysis 
revealed that their psychologists were com- 
posed of two groups: one of which pre- 
ferred to deal with dynamic inferences, and 
the other with inferences related to the kind 
of psychopathology present. 

Secondly, in neither study were controls 
utilized. That is, there was no way of tell- 
ing to what the O sorts were 
“stacked” in that some of the items may 
have applied to all or most patients in a 
particular age group while others may have 
had little or no applicability. The likelihood 
that this actually occurred at least in Little 
and Shneidman’s study is strongly suggested 
by an examination of the 24 


degree 


items these 
authors reported as being ranked highest 
and lowest on the O-sort continuum, All 
12 of the items ranked highest in appli- 
cability dealt with pathological aspects of 
the patient (for example, “he is in the early 
stages of paranoid schizophrenia”). Con- 
versely, all 12 of the items ranked lowest 
had positive connotations (“he is not a par- 
ticularly sick individual”). One could legiti- 
mately ask whether the psychologists could 
not have attained almost as high correla- 
tions if they based their rankings on just 
the knowledge that the S was a patient, 
without any test data available. 

In addition to these two failings of the 
studies by Fisher and by Little and Shneid- 
man, two the 
These were: 
(a) only clinical psychologists with a great 


limitations 
applicability of their results 


more restricted 


deal of experience and reputations as “ex- 
perts” were employed in both pieces of 


Thus, 


ing as to the degree of predictive accuracy 


research. no evidence was forthcom- 
of clinicians of less experience and reputa- 
tion. (hb) Since only one heterogeneous O 
sort was used, 
could be made as 
psychologists’ 


only an over-all 


statement 
to the effectiveness of 


appraising 


perse mality via 
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projective methods. No data were made 
available as to their relative effectiveness 
for determining dynamics, defense mechan- 
isms, overt behavior, diagnosis, etc. In the 
present investigation, an attempt will be 
made to overcome the shortcomings and 
limitations of these two pioneer attempts. 


HyPpoTHeses 


Hypothesis I, There is a positive rela- 
tionship exceeding chance agreement be- 
tween the evaluations of clinical psychol- 
ogists working with the projective test mate- 
rial of patients and the evaluations of psy- 
chiatrists working with the same patients in 
psychotherapy. 

A number of prior studies investigating 
the validity of projective techniques have 
reported a great deal of variability in the 
ability of different clinical psychologists 
who acted as participants (Chamber & 
Hamlin, 1957; Samuels, 1952; Symonds, 
1955). Yet there has been a lack of investi- 
gation as to what the crucial differences are 
between those clinicians whose evaluations 
correlate well with the criterion measure 
and those whose correlations do not. This 
study will investigate differences in terms 
of two variables that should be pertinent in 
this regard. Hypothesis II is concerned 
with the experience variable. The expecta- 
tion of differences here is based on the prin- 
ciple that in any endeavor requiring complex 
skills, performance will improve with ex- 
perience. 


Hypothesis IT, 


The degree of positive 


relationship between psychologists’ and psy 


chiatrists’ evaluations 
function of 


ogist 1S; 


will be, in part, a 
how experienced the psychol- 
the greater his experience, the 
greater his accuracy in the 
patient 


appraising 


concerned with the 
personal analysis variable. The expectation 
that there will be a difference in the ability 
of those psychologists who have ufdergone 
Freudian psychoanalysis and those psychol- 
ogists who have not, in making projective 
test evaluations, is based on the following 
rationale. The results of a number of 
studies have demonstrated that a positive 


Hypothesis Ill is 


relationship exists between lack of insight 
into self and lack of insight into others 
( Frenkel-Brunswick, 1951 ; Goodman, 1952; 
Norman, 1953; Sears, 1936). Since projec- 
tive test evaluations involve gaining insight 
into others, it would follow that the degree 
of accuracy with which psychologists ap- 
praise a patient will, in part, depend on 
the degree of insight that they have into 
themselves. Experimental findings support- 
ing this idea have, in fact, been reported 
by Mintz (1955) and Filer (1951). Since 
the therapeutic process in Freudian psycho- 
analysis centers upon the development of 
self-insight (Bibring, 1954), psychologists 
who have received such treatment would be 
expected to show greater accuracy in their 
appraisals, 

During the course of a Freudian psycho- 
analysis, self-insight is developed at many 
different levels. According to Bibring, these 
include the patient’s understanding that he 
reacts in typical ways to typical situations ; 
that certain of his attitudes, which at first 
appeared to him unrelated, are in fact re- 
lated to each other; that certain reaction 
patterns form a characteristic sequence ; and 
that he is motivated by specific unconscious 
ideas and feelings. l'reudian psychoanalysis 
aims at an understanding of all of these, 
but the understanding of unconscious mate- 
rial plays the central role ( Bibring, 1954). 
Thus, a clinical psychologist who has under- 
gone such treatment should be at a particu- 
lar advantage in his projective test evalua- 
tions when unconscious forces of the patient 
need to be appraised. [specially sensitive to 
forces in himself, he would be ex- 
pected to be 


these 
sensitive to these 
Since some of the areas 
evaluated in this 


specially 
forces in others. 
to be research are con- 
cerned with unconscious aspects of person- 
added reason for 


anticipating differences in terms of the per- 


ality, there is thus an 


sonal analysis variable. 
Hypothesis 
relationship between psychologists’ and psy- 


The degree of positive 


chiatrists’ evaluations will be greater for 
have 
Freudian psychoanalysis than for those who 
have not. 


those psychologists who undergone 


The reason for not hypothesizing 


as great a degree of positive relationship 


VALIDITY 


for those psychologists who have received 
other forms of treatment as for those who 
have undergone Freudian psychoanalysis 
rests on my taking seriously a view of Bib- 
ring and thus my wishing to test one impli- 
cation of this view. Bibring writes that one 
of the differences between Freudian psycho- 
analysis and the various types of dynamic 
psychotherapies derived from it is that in 
the latter there is “a general trend to shift 
the emphasis from insight through interpre- 
tation toward ‘experimental’ manipulation, 
that is learning from experience seems to 
become the supreme agent rather than in- 
sight through interpretation’ (Bibring, 
1954, p. 766). 

Hypothesis IV is concerned with the in- 
creasing agreement between the psychia- 
trists’ successive evaluations of his patient 
and the evaluations of the clinical psychol- 
ogists. This bears upon the practical value 
of having patients who are entering psycho- 
therapy tested beforehand. 

I could find only one study in the litera- 
ture bearing on this issue. Siegel (1948) 
compared the degree of agreement between 
two psychiatric diagnoses, made one year 
apart, and the diagnosis made on the basis 
of a Rorschach. She found 88.5% agree- 
ment between the Rorschach and the second 
psychiatric diagnosis, compared to only 
61.5% agreement between the Rorschach 
and the first psychiatric diagnosis. Only 
one psychologist participated in the study 
and only the area of diagnosis was investi- 
gated so that further study of this question 
is very much in order. 

Hypothesis IV. The degree of positive 
relationship between the psychologists’ and 
psychiatrists’ evaluations will be, in part, a 
function of the length of time the psychia- 
trist has seen the patient. There will be 
an increasing correlation between the psy- 
chiatrists’ evaluations of their 
patients and the evaluations of the clinical 
psychologists. This is equivalent to saying 
that, as the psychiatrists’ behavior samples 
are enlarged, their judgments become more 


successive 


valid, and, as the psychiatrists’ judgments 
become more valid, they converge upon the 
test evaluations. 
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Hypotheses V and VI are concerned with 
interaction effects between the variable 
“length of time the psychiatrist has seen 
the patient before making the evaluation” 
and the variables of experience and personal 
analysis. The expectation that the latter two 
variables will affect the degree to which psy- 
chologists can discern the hidden core of 
personality, thus perceiving aspects of the 
patient that are not clinically apparent for 
some time, is based on essentially the same 
rationales as those underlying Hypothesis IT 
and Hypothesis ITT. 

Hypothesis V. The greater the experience 
of the psychologist, the more pronounced 
will be the increasing correlation between 
his evaluation of the patient and the psy- 
chiatrist’s evaluations of the 
patient. 

Hypothesis VI. There will be a more pro- 
nounced increasing correlation between 
judgments by those psychologists who have 
undergone I'reudian psychoanalysis and the 
psychiatrists’ successive evaluations than be- 
tween judgments by those psychologists 
who have not undergone such treatment and 
the psychiatrists’ successive evaluations. 


successive 


Metnops Usep 
Selection of Cases 


The Ss whose projective test material was to be 
evaluated were 10 young adult males between the 
ages of 17 and 22 who committed crimes that 
brought them to the Court of Special Sessions of 
the City of New York. From here, they were 
either referred directly by the judge or through 
the probation department to the psychiatric clinic 
attached to the court. They were chosen from 
among the total patient population of the clinic in 
the following manner. As soon as it was decided 
to embark upon this project, each new patient seen 
at the clinic who was judged suitable for once- or 
twice-a-week psychotherapy was considered a po- 
tential research case. This meant that he had to 
have administered to him the various projective 
techniques that were to make up the research test 
battery prior to his beginning treatment 
decided that for the purpose of 
should administer all the 
potential research cases 


It was 
consistency, I 
projective tests for 

This procedure was followed for many more than 
10 Ss since it was anticipated that some of the Ss 
would not remain in therapy long enough so that 
an extended psychiatric appraisal could be made. 
Thirty-five was decided the 


upon as 


minimum 


j 
- 
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number of therapy sessions that had to be com- 
pleted for the patient to be considered a research 
case. This caution turned out to be fully justified 
inasmuch as about 25 persons had to be considered 
as research potentials before 10 could be found who 
completed the minimum number of psychiatric 
sessions. The others either refused to continue 
treatment at some point before 35 sessions or else 
were rearrested and given jail terms so that 
therapy could not continue. 


The Projective Test Battery 


The projective test battery administered to the 
research consisted of the following tech- 
niques : 

Rorschach. Administered in accordance with the 
Klopfer technique (Klopfer & Kelley, 1942). 

Thematic Apperception Test. Administered in a 
manner similar to that outlined by Murray (1943) 
with the following exceptions: (a) There was no 
time limit imposed for each story. (>) Only one 
testing session was held, this following the proce- 
dure described in Murray’s “first session.” (c) The 
following ten cards were presented to each S 
1, 2, 6 BM, 7 BM, 7 GF, 8 BM, 9 GF, 12 M, 
13 MF, and 18 GF. In addition, on the basis of 
my clinical judgment regarding what other cards 
would yield significant material in that particular 
case, between two and nine additional stories were 
obtained from each S from among cards 3 BM, 4, 
5, 6 GF, 9 BM, 10, 13 B, 17 BM, and 18 BM 

House-Tree-Person Drawings. Administered in 
accordance with the Buck (1948) technique except 
for the fact that regular 84 X Il-in. paper was 
used. In addition, inquiry was made about the 
figures drawn and, where it seemed appropriate, 
about the house and the tree as well 

Most-Unpleasant-Concept Test. 
accordance with the Harrower (1952) technique 

The first three of these projective were 
chosen because of their very wide use in clinical 
practice. The fourth (Most-Unpleasant-Concept) 
was included because of the very small amount of 
time needed for its administration which [ have 
found to be amply justified by the richness of the 
data it vields 


cases 


Administered in 


tests 


Clinical Psychologist Evaluators 


Thirty clinical psychologists took part in evalu 
ating the protocols. They 
divided into groups in terms of two variables 

Experience variable—Group I. Ten clinical psy- 
chologists who have had 10 or more vears of pro- 


earned 


projective test were 


experience and who have 
through their supervising, 
and/or writing for their high competence in deal- 
ing with projective test material. The decision as 
to whether the psychologist had earned such a 
reputation was made by me with the concurrence 
of at least 


Group 


testing 


reputations teaching, 


thesis committee 


Ten clinical psychologists who have had 


one member of my 


between five and eight years of projective testing 
experience. Group III. Ten clinical psychologists 
who have had three or fewer years of projective 
testing experience, including their internship. 
Personal analysis variable—Group A. Ten clin- 
ical psychologists who either completed a Freudian 
psychoanalysis or who completed a large part of 
their analysis although still in treatment. This 
group hereafter will be referred to as the “psycho- 
analyzed group,” 
analyzed 
clinical 


and its members as the “psycho- 
"* Group B. Nineteen 
psychologists who either had not under- 
gone Freudian psychoanalysis at all, or, as was 
with three participants, had recently be- 
treatment but had not completed more 
than four months of it. This group hereafter will 
be referred to as the “nonpsychoanalyzed group,” 
and its members as the “nonpsychoanalyzed psy- 


iwlogists 


psychologists 


the case 


gun such 


One psychologist, who had been in 
psychoanalysis for one year and then 
had left, could not qualify as belonging to either of 
the two groups. Thus, her evaluations were not in- 
cluded when the variable of personal analysis was 
considered 


Freudian 


It was noted that for further comparisons, the 
psychologists in Group B could be subdivided into: 
Subgroup B:—nine psychologists who had either 
completed a course of individual treatment other 
than Freudian psychoanalysis or who had com- 
pleted a large part of such treatment. The mem- 
bers of this group varied in terms of the orienta- 
tion of their therapists and the frequency of their 
sessions. Some described their treatment as once- 
or twice-a-week psychotherapy and others as Sul- 
livanian convenience, this group 
hereafter will be referred to as the “psychotherapy 
group. 


analysis. For 


Subgroup B:—nine psychologists, among 
whom seven had received no amount of individual 
psychotherapy and two who had just begun a 
Freudian psychoanalysis. This group hereafter will 
nontreated group.” One of 
the nineteen psychologists in Group B had received 
some form of treatment other than Freudian psy- 
choanalysis tor a period of between six and sixteen 
months during the participation in 
This was thought to be too short a 
time for him to qualify for Subgroup 
B, and too long a period of time for him to qualify 


be referred to as the 


course of his 
the research 


period of 


1 


* Five members of tl 
Freudian 


is group had completed a 
psychoanalysis, were in the sixth 
of their analyses, one in his fifth year, and 


two in their third vear. 


two 


year 


group had completed 
four members had completed 
at least three treatment. In terms of the 
orientation of their therapists, six of the psychol- 
received treatment 
the interpersonal 


5 Five members of this 


their treatment and 


vears ot 


from a member of 
recc¢ ived 
o was described as 
f Stekel, and two who described their therapists 
in orientation 


gists had 
school, one 
} 


reatment 


a therapist w a follower 


as eclectic 


| | 
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for Subgroup B:. Thus, when these two subgroups 
were compared and when a three-way comparison 
was made, the evaluations of this psychologist 
were not included. An approximately equal pro- 
portion of participants from each of the three ex- 
perience groups was included in Groups A and B 
on the one hand, and Subgroups B, and B: on the 
other. This meant that when either the experience 
variable or the personal analysis variable was to 
be considered separately, the other 
stant. 


was held con- 


Distribution of Cases Among Psychologists 


Each psychologist was asked to evaluate the 
protocols for two of the ten patients. Thus, 60 
psychological In order to 
keep equal the number of psychologists doing each 
case, the protocols for each patient were evaluated 
by six psychologists 


evaluations were made 


In order for the most meaningful comparisons 
to be made in regard to differences between psy- 
chologists at the three experience levels, each case 
was assigned to two psychologists from Group I, 
two psychologists from Group II, and two psy- 
chologists from Group III. In regard to the per- 
sonal analysis variable, at least one psychologist 
representing Group A and Group B and at least 
one psychologist from Subgroup B, and Subgroup 
B: participated in each case. 

For appraising each case the psychologist was 
provided with a copy of the patient’s Rorschach, 
TAT, and drawings of house, tree, persons, and 
most unpleasant concept. In addition, the psy- 
chologist was informed of the patient’s age, sex, 
race, nationality, the number and sex of siblings, 
and whether or not the patient was currently living 
at home. A brief statement was also made about 
the patient’s upbringing in terms of who reared 
him (one or both parents, or some other person). 
The psychologists were also told that all the cases 
came from the files of a psychiatric clinic attached 
to the court. Under ideal conditions, the latter 
fact should not have been made known to the psy- 
chologists in a study such as this. However, some 
of the participating psychologists who were closely 
with me knew the source from which 
the cases were being drawn. Thus, to equalize the 
conditions under which the various psychologists 
operated, this fact was revealed to all, and a con- 
trol had to be instituted (to be described later) to 


associated 


determine if this knowledge influenced the results. 


Criteria for lalidation 


The 


evaluations 


validity of the 
determined by correlating 
with evaluations made by the psychiatrist 
the patient. In addition to his 
patient in therapy, the psychiat 
evaluation was free to utilize ma 
clinic social worker who had 
parents of the 


degree of psychologists’ 


was them 
treating 


contact with the 


patients, t I of the 
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take psychiatrist, and the probation report on the 
patient. In no instance, however, were the results 
of the psychological given at intake made 
available to the psychiatrist. This criterion evalua- 
tion took place at a point after 35 or more therapy 
sessions. 


tests 


Background of the Participating 
Psychiatrists 
Since each 
chiatrist, 10 
had between five 
perience 


patient was 


psycl 


ifferent 


They 


seen by a ¢ 


participated 


psy- 
had 
years of psychiatric ex- 
residents, in appoint- 
practice. The mean number 
eight. All the psy- 
completed postresidency psy- 
psychoanalytic had 
had at least one and one-half such train- 
ing. The years completed was 
three. They had all received some form of psycho- 


1atrists 
and ten 
either as 
ments, or in private 


clinical 


ot vears of experience 


had either 


chotherapeutic or 


was 
chiatrists 
training or 
years ot 
mean number of 
therapy, either having completed it, or having com- 
pleted two and one-half years or more of it. though 
still in treatment 


The kind of training and personal psychotherapy 


that the psychiatrists received varied. Two of them 
were being trained in Freudian psychoanalysis, one 
at the New York Psychoanalytic Institute and one 
at the Psychoanalytic Institute of the State Uni- 
New York. Four psychiatrists had re- 
ceived training and therapy at the William Alanson 
White Institute, a training 
personal school. The 
had 
anal vtic 

Columbia 


versity of 


center of the inter- 
remaining four psychiatrists 
and therapy at the 
Training and Research at 
This center was under the 
ndor Rado during the time that 
liatrists were trained there and 
Rado’s orientation is no longer considered Freudian 
psychoanalytic (Glover, 1957). 
this were my impressions 
clinic staff 


received traini 

Clink r 
University 
directorship of Sa 


Psycho- 


the four 


with 
gotten from fre- 
that were attended 
From the discussions of 
material, it appeared that the approach of 
the psychiatrists from the Columbia school, both 
in terms l 


Consistent 
View 
quent conterences 
by all the psychiatrists 


Case 


of theory and therapy, was much closer 
approach of the psychiatrists trained at the 
William Alanson Institute than those being 


trained in Freudian ychoanalysis 


to the 


Early Psychiatric Evaluations 

The psychiatrists were also asked to evaluate the 
patients at two earlier intervals during treatment, 
and after 
were also 


18 to 22 sessions. 
correlated with the 
linical psychologists, as was the 
chiatrists. A 
correlations 


com- 

thus 
if there were differ- 
agreement psy- 


ie three intervals. 


between 


+} 


evaluations ot 
criterion eV iluatic 
} 
with ences in the 
cists 
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Method of Evaluation 


The method of evaluation for both psychologists 
and psychiatrists involved the use of Q sorts. The 
evaluator was given six separate sets of state- 
ments, one for each of six areas to be appraised. 
Each set consisted of 30 statements and was dealt 
with separately. The 30 statements were sorted 
into seven groups ranging from most to least ap- 
plicable in accordance with the following distribu- 
tion: one—l, three—2, six—3, ten—4, six—5, 
three—6, and one—7. The rationale for this pro- 
cedure has been outlined by Stephenson (1953). 

In the second and in the final evaluation by the 
psychiatrists, I reminded them to appraise the 
patient in terms of the type of person he was when 
the projective techniques were administered, ex- 
cluding any changes that may have occurred as a 
result of psychotherapy. 

In the appraisals by the psychologists, the pro- 
jective test material was first left with them so 
that they could familiarize themselves with it and 
make whatever notes were necessary for the evalu- 
ation. They were asked to spend about as much 
time looking over the material as they would 
ordinarily devote to looking over case material in 
their clinical practice. Then when they met with 
me for the actual Q-sort evaluation, they did not 
have to spend too much time referring back to the 
raw data. 

During the first and second evaluations by many 
of the psychiatrists, I noted that despite the fact 
that no limit was placed on the amount of time 
they could spend on making their evaluations and 
despite the fact that they were compensated for 
their time, they sometimes rushed through the 
procedure At times, their decisions appeared to 
be made quickly and impulsively rather than as 
the result of reflective thought. Since the third 
and final evaluation of the psychiatrist was to be 
the criterion measure against which the psychol- 
ogist’s evaluations were going to be compared, it 
was felt that the psychiatrists should be  encour- 
aged to spend more time on this final evaluation. 
The following steps were therefore taken. First, 
the third evaluation was allowed to proceed in the 
same manner as the first two evaluations. Next, 
it was explained to the psychiatrist that further 
meetings with him would be necessary in order for 
him to check his third evaluation; that since this 
was to be the criterion evaluation, it should be as 
possible. Then, additional 


accurate as meetings 


were held in which an extended checking proce- 
dure took place for each of the six evaluation 
areas. For every psychiatrist, at least a few shifts 


in ratings were made in each area and it was my 


impression that these shifts were the consequence 


of greater care taken and more time 


spent in 


consideration 


Areas of Evaluation 


Each 


chologist or 


time a patient 
psychiatrist, evaluations were 


was appraised by a psy 


made 
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separately for six areas. The items utilized in the 
six Q sorts appear in the appendix. Each of the 
areas will now be discussed separately : 

Defense mechanisms. The items in this area all 
dealt with the means by which people protect them- 
selves against anxiety generated by intrapsychic 
conflict. To insure for uniformity in approach, all 
the evaluators were instructed to consider as the 
most applicable mechanisms those that 
were most characteristic of the particular patient 
rather than those that were used quantitatively the 
most often. The example of “repression” was 
given, this defense probably used quantitatively 
more often than other defenses by most people. 
However, this was not to be considered 
as one of the most applicable unless it especially 
characterized the particular patient as it would, 
for example, in the case of a classical hysteric 


VM otivating needs and affects. Some of the items 
in this area pertained almost exclusively to un- 


motivating 


defense 


defense 


conscious (for example, those 
referring to castration fears and incestuous long- 


Ings ) 


forces 


Other motivators, however, such as quilt 
heterosexual impulses and_ exhibitionistic- 
voyeuristic needs, are conscious or preconscious in 
and in others. In ap- 
praising this area, all evaluators were instructed 
to ignore the question of whether the motivator 
under consideration 


over 


some persons unconscious 


was conscious or unconscious 
to the patient but to consider only whether and 
to what degree it was motivating him. For ex- 
ample, the item on “feelings of inferiority” was 


meant to apply to a patient if he was either con- 
sciously plagued by feeling less able than other 
people or only aware of 
served as a 


feelings of grandiosity 
compensatory 
feelings. 


which defense against 
the inferiority 

The decision to ignore the level of consciousness 
was made First, it would seem 
much more appropriate for projective testers to 
concern themselves with the which a 
particular was present rather than with 
whether it conscious or not, since the latter 
could be answered more readily by the 
psychotherapist. Secondly, from my contact with 
projective testing, it seems rather doubtful to me 
that the question of whether a patient's strivings 


for two reasons 


degree to 
force 
was 
question 


are conscious or unconscious can be answered on 
the basis of projective tests alone. 

In rating the items in this area, to insure for 
uniformity of purpose, the problem of level (i.e., 
immediacy ) had to be dealt with, for it 
maintained that most of 


also 


could be justifiably these 
motivating forces play a significant role in every- 
development. Thus, the evalu- 


ators were told to give priority to those motivators 


one’s personality 


were influencing the patient at the present 
needs and fears that would reveal themselves 


in current dreams, fantasies, slips of the tongue, 
etc. Other 


hye the 


motivating forces, even if judged to 
of the patient's illness, were to be 


they 


core 


as secondary if were judged to 


| 
+... 
tin 
considered 


have no direct discernible influence on the patient’s 
behavior at the present time 
Character 


evaluators 


traits. In evaluating this area, all 
that the 30 statements 
contained therein were all meant to apply to deeply 
embedded and long-lasting 
patient and not to 


traits 


were instructed 
characteristics of the 
transient or easily modifiable 

Diagnosis and 
area were of 


The statements in this 
Some dealt with 
acter structure and others dealt with the 
matic picture. The 


terms ot 


yaiptoms 
two types char- 
sympto 
latter items were stated in 
the patient's vulnerability to the particu- 
lar symptom or syndrome. The rationale for this 
type of formulation is that it seems more useful 
and more appropriate for a projective tester to call 
attention to the patient’s potential for developing 
“psychosomatic “sexual perversion,” 
“schizophrenia,” “anxiety 

than to decide whether or not such a 


ptoms,” 


sexual impotency,” 
states,” etc., 
State existed at the particular time the patient was 
tested. It is more useful in the that the 
therapist can usually quite easily discern for him 
self the presence of And 
it is often the 
actual manifestation of these disorders depends on 
external circumstance or the patient’s physical con 
dition, in addition to his 
state 


sense 
symptom or syndrome 


more appropriate because ver) 


internal psychological 
factors the psychologist may not be aware of 
Interpersonal behavior 


statements 


The question of whether 
a patient’s overt behavior with 
other persons, particularly with as much specificity 
as is demanded in the items in this O sort, could 
be made from projective test material has been de- 
bated among clinical psychologists. It was because 
of the this that this area 
was included in the hope that the results obtained 
might lend support to one or the other viewpoint 
In appraising this area the psychologists and psy- 
chiatrists were asked to be guided only by how 
they thought the patient typically acted, disregard 
ing entirely the question of motivation 


about 


disagreement on issue 


Infancy and childhood as perceived by the pa- 
tient. This area, like the last mentioned, deals with 
material over which clinical psychologists are in 
as to its predictability from projec 
tive tests, a difference of opinion which the results 
of this study 
the evaluators 


disagreement 


might bear upon. I emphasized to 
when they were making their ap- 
praisals in this area that these items referred only 
to how the patient perceived parental figures during 
infancy and childhood and not to how the parental 
figures actually behaved. As in 
vating needs and affects, the ev 
that thev di rn themselves wit! 
the question of whether these perceptions were 


the area of moti 


luators were told 


10t have to conce 


currently conscious to the patient or, for that 


matter, if thev were ever conscious. The crite 


by which their applicability was to be judged w 


the degree to which these perceptions plaved 


role in influencing the patient's personality devel 


opment 
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Selection of Statements for the © Sorts 

The items used in the O sorts were obtained in 
Several hundr«d statements 
were extracted from psychological reports written 
by experienced clinical 


the following manner 


psychologists for psycho- 


therapists. I then divided them into the six afore- 
mentioned areas. After this they were presented 
to a group of five psychologists who were asked 
to eliminate all items that they considered to be 
vague or ambiguous, inappropriate to the area to 
be evaluated, ar repetitious. I made further 
eliminations and some additions and rephrased 

of the items so that each of the @] sorts 
t contained 30 statements, each of which I 


ged to be stated clearly and concisely 


Definition of Terms 


In order that the statements in the 


O sorts have 
precise and uniform meaning for all evaluators, 
definitions of terms that I thought might still prove 
ambiguous were read to the psychologists and the 
he course of the psychia- 
a number of 
psychiatrist commented 
nbiguity of some term that had 
not been defined. I could not then decide upon a 
definition for that term since it 


with the wav the 


evaluations, 


nstances arose In wl 


about the possible 


might not have 
psychiatrist had im- 

! himself during the one 
or two prior evaluations. Instead, the psychiatrist 
asked | ‘ ig he had ascribed to the 
term during his earlier appraisals and he was then 
told to continue using the term in the 
The 
then 


com ided 


was what n nir 
same way 
psychiatrist's definition was written down and 
ncluded among the read to the 
six psychologists evaluating that particular case. 


definitions 


Control 1 


In order to determine if the amount of agree- 
ment between the psychologists and the criterion 
was greater than chance, some estimate of chance 
agreement first had to be determined. It would 
have been incorrect to that chance agree- 
ment would lead to zero correlations, for it is pos- 


sible that some of the 


assume 
items in the QO sorts apply 
young adult males while 
or no applicability. In order to 
allow for this possibility, the following procedure 


to the great majority of 


others have little 


was employed. Each psychiatrist was asked to do 
Q-sort evaluations for each area on some other 
young adult male patient whom he was seeing or 


seen in therapy in his private prac- 
hom he felt at least as knowledge- 
was of the clinic patient at the time of 
e third evaluation. This made it possible to cor- 


te the Q-sort evaluations 


nd about w 


for each of the six 
evaluated the projective test 
a particular clinic patient with the psy- 
chiatrist’s evaluations of the matched control case. 


ese correlations then gave 


a measure of chance 


a 
= 
i 
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Control 2 


The fact that the clinic patients were all court 
cases—information that the psychologists were 
aware of before making their evaluations—while 
the control patients were not court cases, could 
have led to spuriously high correlations. In order 
to determine if the psychologists were aided by a 
significant degree of stereotype accuracy, an addi- 
tional control had to be instituted. A sample of 
10 of the 30 psychologists were asked to evaluate 
a third case. In order that the various subgroups 
be represented, the 10 psychologists included at 
least two from each of the experience groups and 
from each of the personal analysis groups. In 
evaluating the third case, the psychologists were 
misintormed by being told that the person to be 
appraised was a private therapy patient. All 10 
of the psychologists later reported that they a¢ 
cepted the instructions given them and evaluated 
the patient with the set I wanted them to have 


In order to eliminate the effect of practice, three 
of the 10 psychologists did a case under control 
conditions before doing either of the cases under 
experimental Three others worked 
under conditions between the first and 
second case done under experimental conditions 
And four psychologists evaluated the control case 
after both evaluations under experimental condi- 
tions had been completed. 


miditic ms 
control 


\ second kind of balancing was also necessary 
For each psychologist, the case done by him under 
control conditions was the same case as was done 
by one of the other psychologists among the ten, 
under experimental conditions. This allowed for 
a comparison to be made of the same cases evalu 
ated under control and experimental conditions 


Reliability of the Criterion Measure 


In order to measure the reliability of the psy- 
chiatrists’ evaluations, the following procedure was 
instituted. Seven of the 10 psychiatrists, who were 
able to devote additional time, were asked to evalu 
ate some private practice patient whom they had 
stopped seeing at least six months prior but of 
whom they had a good recollection. Then after a 
period of approximately two months, during which 
time they had no further contact with this patient, 
they were asked to again evaluate him. The two 
sets of evaluations were then correlated and a 
reliability arrived at. The psy- 
chiatrists were asked to select a patient they had 
not seen for at least six months since, with a more 


measure of was 


recent patient, there might have been a significant 
amount of forgetting between the first and second 
evaluations. But once six or months had 
elapsed, it seemed: unlikely that the further interval 
of two months would lead to a significant amount 
of additional forgetting. For this reason, the psy 
chiatrists could not use their clinic patients for the 


reliability check they had verv 


more 


since recently 


terminated treatment with them 


RESULTS 


In order to correlate the quantified assess- 
ments of psychologists and psychiatrists, 
the Method-of-Difference correlational for- 
mula, which is based on the equality of the 
variances and means of the two distribu- 
tions being correlated, was used. The latter 
conditions are imposed by the forced QO 
distribution. 

For every psychologist, his evaluations 
for each of the two cases that he did were 
correlated with the psychiatrist’s four sets 
of evaluations (first, second, third, and cor- 
rected third evaluation). This procedure 
was followed for each of the six areas ap- 
praised. Thus a total of forty-eight correla- 
tions were computed for each of the thirty 
psychologists, making 1440 correlations in 
all. Each of these correlations was then 
transformed into a z equivalent. For each 
(one for 
each case) were averaged and the average 
was treated as a score. Thus, for each of 
the 30 psychologists there were four mean 


area every psychologist’s two 


scores, representing the degree of agreement 
with the psychiatrist’s first, second, third, 
and corrected third evaluations, for each of 
the six areas. In order to arrive at a meas- 
ure of the psychologist’s over-all agreement 
with the criterion the mean scores for the 
SIX areas were averaged. Thus. there were 
four over-all mean scores for each of the 
thirty psychologists representing the degree 
of agreement with the psychiatrists at the 
time of the latter’s four appraisals. 

In order to test Hypothesis I, the mean 
scores representing the degree of agreement 
with the psychiatrists’ corrected third evalu- 
ations were used since these were judged to 
represent their most considered and there- 
fore most accurate judgments. The over-all 
mean scores and the mean scores for each 
of the six areas for each of the thirty psy- 
The results of 
computations are the seven master 
mean scores presented in Table 1, Column 
2. Column 1 of the same table contains the 


chologists were averaged. 
these 


corresponding r values which represent in 
correlational terms, the average degree of 
agreement between the psychologists and 
the psychiatrists. 


| 


In order to establish a base of chance pre- 
diction with which these experimental mean 
scores could be compared, Control 1 was 
utilized. For each of the 30 psychologists, 
his evaluations for each of the two cases 
appraised were correlated with the psychia- 
trists’ evaluations of the matched control 
cases. These correlation coefficients were 
also transformed into z equivalents. Every 
psychologist’s two z’s (one for each case) 
were averaged for each of the six areas. 
An over-all score was computed in the 
same way as was described above. Then, 
the over-all mean scores and the mean scores 
for each of the six areas of each of the 
30 psychologists were obtained. The result 
was a set of master control means which 
are presented in Table 1, Column 3. These 
represent a measure of chance agreement 
between the psychologists and the criteria. 

Hypothesis I posited that the evaluations 
of the psychologists would agree with the 
evaluations of the psychiatrists to a degree 
significantly greater than chance. This hy- 
pothesis was tested by means of ¢f tests in 
which the means reported in Columns 2 
and 3 of Table 1 were compared. The differ 
ences between the two sets of are 
reported in Column 4 and the ¢ values in 
Column 5. 


scores 


The over-all difference is signifi- 
cant, as are also the differences for Areas 
Il, 111, IV, V, and VI. For Area T, the 
difference is not significant. However, it 
should be noted that if the significance 
criterion was relaxed to the .10 level, this 
difference too would be significant.’ 


*In judging p values in this study, the usually 
accepted criterion of .05 is being taken as indicating 
significance, and it is only when this 
reached that a hypothesis can he 
having received solid support 


level is 
considered as 
However, in a num 
ber of its aspects this study is investigating prob 
lems that have not 
lished research 
to which psycl 
tions 


been dealt with in 
These problems in 


prior pub 
clude the decree 
ologists can make accurate evalua 


from projective test data for a number of 


delineated areas of functioning involving different 
levels of personality, differences in predictive ability 
of psychologists for these areas in terms of ex 
perience and personal analysis variables, and_ the 
ability of psychologists to offer psychotherapists 
information patients t 


not apparent early 


about for these areas 


from therapy sessions. Since 
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Before these results can be taken as sup- 
porting Hypothesis I, the results of Control 
2 must be evaluated. In order to determine 
if there were significant differences between 
the psychologists’ correlations and the psy- 
chiatrists’, when the former group knew that 
the patients were court cases compared to 
when they were not aware of this fact, f 
tests were performed. The ¢ values were all 
extremely small and none were significant.’ 
Thus, it can be assumed that the significant 
positive correlations between the evaluations 
of the psychologists and the psychiatrists, 
reported in Table 1, were not dependent on 
the psychologists’ knowledge that the patients 
who were appraised were court referrals. 
Hypothesis I can therefore be considered as 
supported for the over-all evaluations of 
the psychologists and for their evaluations 


in the areas of Motivating Needs and 
Affects, Character Traits, Diagnosis and 


Symptoms, Interpersonal Behavior, and In- 
fancy and Childhood Perceptions. In the 
area of Defenses, there is a tendency in the 
hypothesized direction 

Hypothesis II posited that the more ex- 
perienced the psychologists, the more accu- 
rately they would be able to evaluate pro- 
jective test material. This was tested by a 
one-dimensional analvsis of variance design 
as presented in Edwards (1950, p. 186), in 
which the three experience groups were 
compared 


The mean for each ex- 


perience group and the summaries of the 


scores 


analyses of variance are presented in Table 


2. The F 


values did not reach one either 


being investigated, it would be 


particularly unfortunate if a Type II error was to 
he made. | his reason, mention shall also be 
made of results that would be significant if the 
significance ion was relaxed to the .10 level 
Such results should be considered as indicating a 
tendency I lar direction and should be 
regarded wit! good degree of tentativeness. They 
should be ht « s providing leads to be 
wed up in future investigations 

‘In this instance, and in all future instances 
where ¢ values and F values are described as in- 
significant without further qualification, it should 
be understood that significance would be 
reached even if the criterion was relaxed to the 
10 level 
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for the over-all differences or for the differ- 
ences in any of the six areas, and thus 
there is no justification for abandoning any 
of the null hypotheses. Therefore, Hy- 
pothesis II received no support from the 
analysis of the data. 

Hypothesis III posited that the psycho- 
analyzed psychologists would evaluate pro- 
jective test material more accurately than 
the nonpsychoanalyzed psychologists. Again, 
the correlations of the psychologists’ evalu- 
ations with the psychiatrists’ corrected third 
evaluations were used. Table 3 presents the 
mean scores for the two groups, the differ- 
ences between the two groups, and the ¢ 
values obtained through tests of signifi- 
cance. The over-all difference between the 
two groups was in the predicted direction 
and significant. In all areas except Area V, 
the differences are again in the predicted 
direction, but they are not significant. How- 
ever, if the significance criterion was re- 
laxed to the .10 level, the differences for 
Areas II, IV, and VI would be significant. 
Therefore, Hypothesis III is supported 


Areas for Experimental 


Cases Based on 


Z-Transformations 


Defenses .16 


Motivating Needs and Affects | .23 


Character Traits 41 
Diagnosis and Symptoms 
Interpersonal Behavior 


Infancy and Childhood 
Perceptions 


Over all 


* Significant at .001 level 
» Significant at .01 level 
® Significant at .05 level 
1 Would be significant if 


criterion was re 


laxed to .10 ley 


TABLE 1 


MEAN SCORES FOR VALIDITY AND MEAN SCORES FOR CONTROL 1 
AND ¢t TESTS FOR DIFFERENCES BETWEEN THEM 


Average Correlations 


when 


the over-all evaluations of the two 
groups are considered. When the areas of 
Motivating Needs and Affects, Diagnosis 
and Symptoms, and Infancy and Childhood 
Perceptions are considered separately, there 
are tendencies in the hypothesized direction. 
For the areas of Defenses, Character Traits, 
and Interpersonal Behavior, no such tend- 
encies are present. 

Hypothesis IV posited increasing agree- 
ment between the psychiatrist’s successive 
evaluations of his patient and the evalua- 
tions of the psychologists. Hypothesis V 
posited that this increasing agreement would 
be greater for the more experienced psy- 
chologists. In order to test both these hy- 
potheses a comparison of validity correla- 
tions using the psychiatrists’ first, second, 
and third evaluations was made for the 
psychologists as a total group and for psy- 
chologists divided into experience groups. 
Correlations were transformed into z equiv- 
alents and subjected to a “repeated measure- 
ments of the same subjects” analysis of 
variance design as outlined by Edwards 


Mean Scores | 


Diff. | ¢ Value 
Experimental 
Cases 


Control 
Cases 


.1567 


.0820 


. 2260 


.0740 


1520 3.918 


.4430 


1653 .2777 4.69" 


= — = — — 
| | 
| 
0747 | 1.714 
) .4293 .3187 .1106 
4 1433 0333 .1100 2.24¢ 
31 .3217 .1770 1447 2.78> 
.28 2867 1417 5.478 


VALIDITY OF EVALUATIONS MADE FROM PROJECTIVE TECHNIQUES 


TABLE 2 


MEAN SCORES FOR THE THREE EXPERIENCE GROUPS AND 
SUMMARIES OF THE ANALYSES OF VARIANCE 


Mean Scores 

Source of | 

Areas | Variance df MS F value 
Group I | Group II | Group III 


Defenses .189 .138 Between 2 0079 | 8 
Within 
Motivating Needs and Affects .211 .201 . 266 Between 2 .0123 . 
Within 27 .0224 
Character Traits .500 .375 Between 2 0401 | 
Within 27 
Diagnosis and Symptoms 447 .398 Between 2 | .0074 
| Within 27 .0204 
Interpersonal Behavior Between 2 0026 | 
| Within 27 .0283 | 
Infancy and Childhood 275 | Between 2 .0172 
Perceptions Within 27 .0286 


Over all Between 


Within 


.0102 


* Error estimate larger than variance estiinate for the effect. 


TABLE 3 


MEAN ScoRES FOR THE TWO PERSONAL ANALYSIS GROUPS 
AND t TESTS FOR DIFFERENCES BETWEEN THEM 


Mean Scores 


Areas ‘ Diff. t Value 
Psychoanalyzed Nonpsychoanalyzed 
Group Group 


Defenses 
Motivating Needs and Affects 
Character Traits 
Diagnosis and Symptoms 
Interpersonal Behavior 
Infancy and Childhood Perceptions 


Over all 


* Significant at .05 level 
>’ Would be significant if criterion was relaxed to .06 level 
¢ Would be significant if criterion was relaxed to .10 level 
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2 
27 || : 
| 
| .300 .195 .105 1.92% 
.502 .390 .112 1.50 
484 391 .093 | 2.03» 
167 ~.056 .75 
392 279 113 | 1.728 
: 328 .260 .068 2.158 


(1950, p. 284). The results of these anal- 
yses are presented in Table 4. 

In regard to over-all differences and for 
Areas I, IV, and VI, the F values are not 
significant so that there is no justification 
for rejecting the null hypothesis. For Areas 
III and V, the F values for differences be- 
tween the three sets of scores are significant 
so that the null hypotheses can be rejected. 
For Area II, the F value would be consid- 
ered significant if the significance criterion 
was relaxed to the .10 level. For the last 
three areas mentioned, f¢ tests per- 
formed. 


were 


For Area II, the only significant ¢ value 
was for the comparison of the first and the 


second set of scores. The mean 


for the 


Defenses 


Between trials 
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TABLE 4 
SUMMARIES OF ANALYSES OF VARIANCE FOR DIFFERENCES IN AGREEMENT WITH PSYCHIATRISTS 


AT THREE TIME INTERVALS AND INTERACTION EFFECT OF THESE DIFFERENCES 
With DIFFERENCES FOR EXPERIENCE VARIABLE 


Source of Variation 


second set was the greater so that the re- 
sults were in the anticipated direction. For 
Area III, significant t values were found 
when the first and third sets of scores were 
compared and when the first and second sets 
of scores were compared. In both instances, 
the mean of the first set was the lower of the 
two so that the results were again in the hy- 
pothesized direction. For Area V, a signifi- 
cant t value was obtained when the first and 
third sets of scores were compared. When 
the second and third sets of scores were 
compared, the ¢ value would be considered 
significant if the significance criterion was 
relaxed to the .10 level. However, in both 
these instances the mean of the third evalu- 
ation was the lower of the two so that these 


2 | d 
Interaction trials X Experience groups + 0095 | 1.56 
Pooled interaction (Error term) 54 .0061 
Motivating Needs and Between trials 2 .0217 2.52¢ 
Affects Interaction trials X Experience groups .0036 
Pooled interaction (Error term) 54 .0086 
Character Traits Between trials 2 .0629 8.50* 
Interaction trials X Experience groups 4 0037 | d 
Pooled interaction (Error term) 54 .0074 
Diagnosis and Symptoms Between trials 2 .0202 1.98 
Interaction trials X Experience groups 4 .0029 d 
Pooled interaction 54 .0102 
Interpersonal Behavior Between trials 2 .0250 3.79% 
Interaction trials X Experience groups 4 .0034 qd 
Pooled interaction (Error term 54 .0066 
Infancy and Childhood Between trials 2 .0077 d 
Perceptions Interaction trials X Experience groups + .0022 d 
Pooled interaction 4 


Over all 


Interaction trials 


Significant at .01 level 
> Significant at .05 level 
¢ Would be significant if criterion was relaxed to 


10 level 
4 Error estimate larger than variance estimate for the effe 


Between time intervals 


Pooled interaction (Error term 


2 .0013 d 
Experience groups 4 .0004 d 
t .0020 


| 
Areas df MS F Value 
| 


results are the opposite of what was hy- 
pothesized. 

Thus, Hypothesis IV was supported in 
the area of Character Traits. In the area of 
Motivating Needs and Affects, there was a 
tendency in the hypothesized direction. In 
regard to over-all differences and differ- 
ences in the areas of Defenses, Diagnosis 
and Symptoms, Infancy and Childhood Per- 
ceptions, and Intepersonal Behavior, no 
such tendencies were present. For the last 
mentioned area, there were results that were 
the opposite of those predicted. That is, 
the psychologists showed decreasing agree- 
ment with the psychiatrists’ 
evaluations. 


successive 


Hypothesis V receives no support from 
this study either for over-all differences be- 
tween the three experience groups or for 
differences in each of the six areas treated 
Since none of the interactions 
between experience groups and increases in 
scores at the three time intervals are signifi- 


separately. 
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TABLE 5 


SUMMARIES OF ANALYSES OF VARIANCE FOR INTERACTION EFFECT OF DIFFERENCES IN 
AGREEMENT WITH PSYCHIATRISTS AT THREE TIME INTERVALS AND DIFFERENCES FOR 
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cant (Table 4), the more experienced psy- 
chologists were not increasingly more accu- 
rate than the less experienced. 


Hypothesis VI posited that the psycho- 
analyzed psychologists would show a greater 
degree of increasing agreement with the 
psychiatrists’ successive evaluations than the 
nonpsychoanalyzed. Again, a “repeated 
measurements of the same subjects” anal- 
ysis of variance design was employed; this 
time the two variables were the increase in 
scores at the three time intervals and the 
two personal analysis groups. The relevant 
results of these analyses are presented in 
Table 5. The interaction term is the appro- 
priate one for judging differences between 
the two groups. In regard to over-all differ- 
ences, the F value is not significant. For 
Areas II, III, IV, and V, the F values for 
interaction are not significant so that there 
is no reason to reject the null hypothesis. 
In Area I, the F value is significant, indi- 
cating that the null hypothesis can be dis- 


PERSONAL ANALYSIS VARIABLE 


Areas 


Defenses 


Motivating Needs and 
Affects Pooled interaction 


Character Traits 
Pooled interaction 
Diagnosis and Symptoms 
Pooled interaction 


Interpersonal Behavior 
Pooled interaction 


Infancy and Childhood 
Pe:ceptions Pooled interaction 


Over all 


* Significant at .05 level 
> Would be significant if criterion was relaxed to .10 level 
iate for the effect 


e Error estimate larger than variance estin 


Source of Variation df MS 
Time intervals X Personal analysis 2 
Pooled interaction (Error term) 54 
Time intervals X Personal analysis 
Time intervals X Personal analysis 
Time intervals X Personal analysis 


Time intervals X Personal analysis 


Time intervals X Personal analysis 


Time intervals X Personal analysis 
Pooled interaction 


.0192 3.378 
.0057 
2 .0124 1.70 
54 .0073 
2 .0040 
54 .0070 
2 .0015 e 
54 .0098 
2 .0029 
54 .0069 
2 .0208 2.42> 
54 
2 .0028 1.47 
54 .0019 


= 
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carded. In Area VI, the F value would be 
significant if the significance criterion was 
relaxed to the .10 level. Thus, t¢ tests were 
performed for these two areas so that the 
results could be further analyzed. 

For Area I, the psychoanalyzed psychol- 
ogists showed the hypothesized increasing 
agreement between the second and third sets 
of scores to a significant degree. Between 
the first and third sets of scores they would 
have shown the hypothesized significant in- 
creasing agreement if the significance crite- 
rion was relaxed to the .10 level. The non- 
psychoanalyzed psychologists showed no 
such increase. For Area VI, the psycho- 
analyzed psychologists showed significant 
increased agreement between the first and 
second sets of scores, while the nonpsycho- 
analyzed group did not. 

Hypothesis VI, then, receives support for 
the area of Defenses while in the area of 
Infancy and Childhood Perceptions there 
is a tendency in the hypothesized direction. 
In terms of the over-all differences between 
the two personal analysis groups and for 
the areas of Character Traits, Diagnosis 
and Symptoms, Interpersonal Behavior, and 
Motivating Needs and Affects, no such 
tendencies are present. 

In addition to the statistics already pre- 
sented which bear upon the six hypotheses, 
additional statistical work was undertaken. 
One problem that was investigated was re- 
lated to the original grouping of psychol- 
ogists along the personal-analysis dimen- 
sion. The nonpsychoanalyzed group con- 
sisted both of psychologists who had _ re- 
ceived some form of treatment, other than 
Freudian psychoanalysis (Subgroup B,), 
and of those who had received no form of 
treatment (Subgroup B.). Comparisons of 
these two subgroups were made regarding 
the amount of agreement between their 
evaluations and the corrected third evalua- 
tions of the psychiatrists to determine if, in 
fact, thev performed in the similar way that 
the original grouping implicitly suggested 
that they would. The mean scores for both 
groups and the ¢ values for the differences 
between the means were derived. None of 
the ¢ values were significant, indicating that 
there is no reason to reject the null hypoth- 


esis regarding the differences between the 
two groups. Thus, the decision to consider 
these two subgroups together in comparing 
them with the psychoanalyzed psychologists 
is supported empirically. 

In order that three-way comparisons of 
the psychoanalyzed group (Group A), Sub- 


group B,, and Subgroup B, could also be 


made, the mean scores for each of these 
three groups were computed. The scores 
for Group A were higher than the scores 
for Subgroup B, and Subgroup B, for over- 
all differences and for Areas IT, III, IV, 
and VI. However, when the differences 
were evaluated statistically, because of the 
breakdown of the psychologists into three 
rather than two groups, the differences be- 
tween the psychoanalyzed and nonpsycho- 
analyzed psychologists were less apparent. 
When a two-way comparison was made, 
the over-all difference was significant ; when 
a three-way comparison was made, it was 
not. However, it should be noted that if 
the significance criterion was relaxed to the 
.10 level, this difference, too, would be 
significant. Although when a two-way com- 
parison was made it was stated that this 
relaxation of the significance criterion 
would result in significant differences for 
Areas IT, TV, and VI, in the three-way 
comparison this would only be true for 
Area I] 

Other statistics of interest center about 
interpsychologist reliability. In order to de- 
termine the degree of agreement between 
psychologists evaluating the same case, the 
same formula was employed as was utilized 
in computing validity coefficients. Each psv- 
chologist’s set of evaluations (one for each 
of the six personality areas) were corre- 
lated with the other five psychologists ap- 
praising the same case. This yielded 15 

correlations per case or a total of 
150 sets of correlations for all 10 cases. 
The correlation coefficients were trans- 
formed into s equivalents, means computed 
and these mean <’s transformed back into 
correlation The coefficients 
were .27 for Defenses, .25 for Motivating 
Needs and Affects, .44 for Character Traits, 
44 for Diagnosis and Symptoms, .21 for 
Interpersonal Behavior, .38 for Infancy and 


sets of 


coefficients. 


Childhood Perceptions, and .34 over all. 
These coefficients represent the average de- 
gree of agreement among psychologists who 
evaluated the same projective test material. 

Just as the psychologists’ validity coeffi- 
cients could be compared for the different 
experience groups and for the two personal 
analysis groups, so could their reliability 
The first 
problem to be investigated in this regard 
was whether degree of experience played a 
part in the amount of interpsychologist 
agreement. Did the more experienced psy- 
chologists more among themselves 
than the less experienced psychologists ? 
The correlations of the two psychologists 
from Group I who did the same case, those 
from Group IT, and those from Group III 
were pertinent in this regard. Since there 
were 10 cases, 10 sets of correlations for 
each of the three experience groups were 
compared, 


coefhcients also be compared. 


agree 


These comparisons were made 
by means of a one-dimensional analysis of 
variance design. None of the F values 
either for over-all differences or for any of 
the six areas considered separately were 
significant. Thus, there is no evidence from 
this research to suggest that the degree of 
experience of a psychologist plays a role in 
determining how much his evaluation of a 
case will agree with another psychologist’s 
evaluation who is approximately at the same 
level of experience. 

A second and even more crucial question 
was asked regarding interpsychologist reli- 
ability in relation to the experience dimen- 
sion. Do the more experienced psychologists 
agree more among themselves than they 
agree with the less experienced psychol- 
ogists? In order to answer this question, 
only the evaluations of the 20 psychologists 
from the two extreme experience groups 
were considered. For each of the 10 cases, 
the correlations, representing agreement of 
each of the psvchologists in Group T with 
both of the psychologists in Group ITI were 
utilized. For all 10 cases, this amounted to 
40 sets of correlations. 
computed and 


Mean scores were 
compared with the mean 
scores representing agreement among the 
with each 
made in 


most experienced psychologists 


other. 


These comparisons were 
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tests, and none of the differences were sig- 
nificant. Thus, this study offers no support 
for the idea that highly experienced psy- 
chologists will agree more among themselves 
than they will agree with less experienced 
psychologists in projective test evaluations. 

To study interpsychologist reliability in 
relation to the personal analysis dimension, 
comparisons were made in the following 
way. The evaluations of every psycho- 
analyzed psychologist were correlated with 
the evaluations of every other psycho- 
analyzed psychologist who did the same 
Similarly, the evaluations of every 
nonpsychoanalyzed psychologist were corre- 
lated with the evaluations of every other 
nonpsychoanalyzed psychologist who did the 
same case. This produced a total of 12 sets 
of interpsychologist correlations among the 
psvchoanalyzed group and 35 sets of inter- 


case. 


psychologist correlations among the non- 
psychoanalyzed group. These 
were transformed to 2 equivalents, means 
were computed for each of the two groups, 
and differences were tested by ft tests. The 


correlations 


over-all difference between the two groups 
was significant, as were also the differences 
for the areas of Defenses, Character Traits, 
and Infancy and Childhood Perceptions. In 
the area of Motivating Needs and Affects, 
the difference would have been significant 
if the significance criterion was relaxed to 
the .10 level. In all four of these individual 
areas, as for over-all differences, the mean 
scores for the psychoanalyzed psychologists 
were the higher. In the areas of Diagnosis 
and Symptoms and Interpersonal Behavior, 
there were no significant differences be- 
tween the two groups 

Another type of reliability score that can 
be reported is that of each of the seven 
psychiatrists who evaluated a private patient 
at two intervals, not having seen the patient 
during the interim. The two evaluations of 
each psychiatrist were correlated with each 
other and the coefficients were transformed 
into < equivalents. Means were then com- 
puted for the N of seven, and these in turn 
were transformed back into correlation co- 
efficients. The coefficients were .70 for De- 
f Motivating Needs 


tenses, 64 for 
Character Traits, .79 


Affects, .76 for 


and 
for 
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Diagnosis and Symptoms, .70 for Inter- 
personal Behavior, .76 for Infancy and 
Childhood Perceptions, and .73 over all. 

The final data to be reported relate to in- 
dividual differences in validity coefficients 
for the six psychologists who evaluated the 
same case. The correlations of each psy- 
chologist with the criterion were utilized for 
this purpose. The spreads of these correla- 
tions for each case were determined, the 
term “spread” referring to the difference 
between the highest and lowest of the six 
correlations for a particular case. Then the 
means of these spreads were computed. 
These means were .45 for Defenses, .45 for 
Motivating Needs and Affects, .42 for 
Character Traits, .42 for Diagnosis and 
Symptoms, .69 for Interpersonal Behavior, 
and .44 for Infancy and Childhood Percep- 
tions. 


DISCUSSION 


An extensive and detailed discussion of the 
findings of this study can be found elsewhere (Sil- 


verman, 1958). Only the most important issues 
will be dealt with here and in some instances in 
summary form 


The findings demonstrating that the group of 30 
psychologists was able to appraise projective test 
material with accuracy 
greater than would be expected by chance, while 
not sufficient for great rejoicing on the part of 
clinicians, do have some importance. For the only 
personality areas in which projective tests, used 
holistically, have been adequately demonstrated to 
have even this much validity in the past, have 
been 


a degree of significantly 


those of Diagnosis and, to a lesser degree, 
Character Traits, as these areas have been defined 
in the current investigation. The results of the 
current research support the findings for 
two areas. In addition, the current research has 
demonstrated that clinicians using projective tech- 


these 


niques can evaluate patients to a degree greater 
than chance in three additional areas. In the 
of Motivating Needs and Affects and Infaney and 
Childhood Perceptions, these results are particu- 
larly noteworthy. For, as far as I could discern 
from published research, this is the first time that 
it has been demonstrated in a controlled, quantified, 
and objectively evaluated study that clinical psy- 
chologists utilizing projective techniques can make 
inferences about the 


areas 


underlying motivating forces 
in patients that are congruent with the inferences 
made by psychiatrists utilizing data 
psychotherapy. The positive results in the areas of 
Interpersonal Behavior and Infancy and Childhood 


revealed in 


Perceptions are noteworthy in another wavy since. 
for these two areas, many psychologists, including 


a number participating in this study, stated that 
they did not think accurate appraisals based on 
projective tests could be made. 

By usual standards the mean validity coefficients 
reported in Table 1 for the six areas would be 
classified as ranging from “low” to “moderate.” 
However, such standards would have little mean- 
ing for this study since they are based on the 
that correlation is .00 and 
maximum correlation is 1.00. In this study, neither 


of these assumptions is justified. 


assumption chance 
The correlations 
attained by Control 1 (Table 1) represent chance 
agreement and all of are above .00. These 
“true” bases should be kept in mind in judging the 
size of the mean validity correlations, though it 
noted that for five of the areas, the co- 
ethcients are low 


these 


can be 


In judging the size of the validity coefficients, a 
second consideration is the actual maximum above 
which no psychologist’s correlations could be ex- 
pected to go 00 could not be taken as 
chance correlation, 1.00 cannot be taken as the true 
maximum for this would imply that the criterion 
measure was completely valid. 


Just as 


This, of course, no 
one, least of all the psychiatrists themselves, would 
claim. Just what the “true” maximum might be is 
extremely difficult to estimate, for by what could 
we judge the accuracy of the psychiatric evalua- 
tions? However, one aspect of the validity of the 
psychiatrists’ evaluations that was measured was 
the reliability of their judgments. While reliability 

small aspect of validity, it does set a 
limit beyond which validity The intra- 


cannot go 


psychiatrist reliability coefficients reported earlier 
can be used to estimate indices of reliability. These 
indices (which are the square roots of the intra- 


psychiatrist reliability coefficients), can be consid 


upper limits beyond 


ered as approximating the 
which the psychologists’ evaluations for a particu- 


lar area could not go. The mean indices were .84 
for Defenses, .80 for Motivating Needs and 
Affects, .87 for Character Traits, .89 for Diag- 


nosis and Symptoms, .84 for Interpersonal Be- 
87 tor Infancy and Childhood Perceptions, 


and .85 over all 


havior, 


In judging the validity coefficients, the reader 
should also keep in mind the following artifacts 
that may have lowered them so that they represent 
in underestimate of the accuracy with which psy- 


chologists are actually able to evaluate projective 


test iterial 

1, Unlike the usual clinical procedure, the psy- 
chologists did not test the patient whom they were 
to evaluate, but had to depend on the notes of 


2. Failure to an intelligence test such as 
the Wecl sler- Be 
} } y 


NOLOGIStS 


nclude 
llevue in the test battery also made 


different from what it 


ve prevented them from operating with their 
usual effectiveness. The oversight of not including 


a highly structured test such as this mav have 


costly 


id particularly 


consequences in the 


study since two of the areas to be evaluated dealt 
with the subject’s actual manner of coping with 
the world (Character Traits and Interpersonal 
Behavior ) 

3. In validation research, the participating psy- 
chologists, by necessity, must interpret the test 
material independently of other sources of data. In 
clinical practice, on the other hand, such interpre- 
tations can be made in the context of information 
gathered from interviews or case history abstract 
The advantage of having such data available, 
carrying with it the implication that more justice 
could be done to the projective test materia! itself, 
has been elaborated upon by Schafer (1954) 

4. The use of Q sorts for evaluation handi- 
capped the psychologists in a number of ways 
For one thing, this was a new and different way 
of ‘recording psychological evaluations for many 
of the psychologists and their unfamiliarity wit! 
the procedure may, in itself, have impaired the 
effectiveness with which they operated. Secondly, 
unlike the psychological report, the Q-sort proce- 
dure compelled the evaluators to appraise a great 
many aspects of the patient in question, regardless 
of whether or not they felt the test material yielded 
pertinent data. Moreover, while in a psychological 
report interpretations can be made with varying 
degrees of certainty by qualifying some statements 
as high level inferences and others as speculations, 
there is no place for this with O sorts. Items are 
only judged in terms of the importance they have 
in the patient’s personality and not in terms of 
how confident the evaluator feels in judging the 
particular item. And finally, just as the forced- 
choice aspect of Q sorts may have compelled the 
psychologist to go out on a limb, they conversely 
limit him in those aspects of the patient that he 
can appraise 

5. The fallibility of the psychiatrists must have 
contributed in some measure to the fact that the 
correlations were not higher than they were. One 
way of reducing errors caused by such variables 
as countertransference distortions might be to use, 
for the criterion measure, the pooled evaluations 
of a group of psychotherapists, each of whom is 
presented an account, as close to verbatim as pos- 
sible, of the actual treatment sessions as well as 
additional data available such as results from 
social service investigations. A procedure similar 
to this was actually employed in the studies of 
Fisher (1952) and Little and Shneidman (1955) 
and the increase in validity coefficients was marked 
The adequacy of the criterion measure might also 
be increased by enlisting the cooperation of more 
highly experienced and skilled psychotherapists, 
such as training analysts, by utilizing as Ss _ pa- 
tients who are being seen in private treatment a 
thus would probably be better motivated to produce 
material, and by utilizing patients who are und 


going a more intensive and extensive type of psy 
chotherapy, such as psychoanalysis 

6. Semantic confusion seemed to have caused a 
large segment of apparent disagreement betwee 
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psychologists and the criterion. My attempts to 
deal with this problem, which were described 
earlier, were far from fully effective for two 
reasons. First, a number of words which neither 
I nor the psychiatrists believed t 


» be ambiguous 
enough to warrant definitions were shown later to 
have been implicitly defined in different ways by 
different evaluators. This came to light in discus- 
sions with the evaluators after the evaluations had 
been completed. It was of particular interest to 
note that some of the terms implicitly defined dif- 
ferently were nontechnical (“optimistic” and 
“grandiose” for example) and find their way into 
the parlance of people other than psychologists and 
psychiatrists. Thus it appears that this semantic 
problem extends beyond the use of psychiatric 
terminology, and to secure uniformity in under- 
standing, virtually every key word should be 
detined in a study such as this 


Secondly, in some it 


tances, the evaluator did 
not really accept the definition that he was given. 
He would acknowledge the definition by nodding 
his head, but from later discussion it was evident 
that he had lapsed back into thinking of the term 
as he had understood it in the past while doing 
the sorting 

7. Related to this problem of semantic confusion 
was the difficulty caused by the broadness of many 
ot the concepts used. For example, interpersonal 
behavior, such as with 


lrawing from heterosexual 
relations, defying authority, or competing with 
peers, can apply in certain ways to a patient and 
not in others. Sometimes postevaluation informal 
discussion revealed that one evaluator selected cer- 
tain aspects of a patient’s behavior to base his 
evaluation on while ignoring other aspects. If the 
psycl ologist and the psychiatrist happened to tocus 
on ditferent aspects of the patient in responding to 
an item, pseudo-discrepancies between their two 
evaluations were apt to arise. There is no easy 
solution to this problem. All items could be made 
highly specific. Thus, instead of the statement “the 
patient withdraws from sexual contact with those 
of the opposite sex,” a number of different items 
could be specified, one related to sexual contact 
with older females, another to such contact witlr 
younger females, and so on. While this type of 
specificity would greatly reduce the number of 
ways in which the statements could be viewed, it 
is questionable if ghly specific statements 


an be made with ice from material culled 


from psychological 

8. Another probable cause of spurious differences 
between the projective evaluators and the psychia- 
ists was differences in orientation. There was 
much variation within both the psychologist and 
he psychiatris the former, 14 de- 


as predominantly Freud 


7 


scribed their ori 


ian, 6 as predon that of the Interpersonal 


School, and 10 stated their position was eclectic, 
with both Freudian and “interpersonal” concepts 
playing a major role in their theoretical thinking 


The psychiatrists included two who were trained 


~ 


li 
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in Freudian psychoanalysis, four who were trained 
at the William Alanson White Institute, and four 
at the Columbia Psychoanalytic Clinic for Train- 
ing and Research. When a psychiatrist of one 
orientation evaluated the same case as a psychol- 
ogist with a different orientation, differences based 
on their orientations were 2pt to occur in the areas 
of Motivating Needs and Affects and Infancy and 
Childhood Perceptions. that | 
have had with various members of the Psychiatric 
Clinic staff have led me to believe that it is in 
these two areas that differences between 
come into sharpest focus 

Note taken of the size of the 
reliability coefficients representing the average de- 


Case discussions 


schools 


should also be 
gree of agreement among psychologists who evalu- 
ated the same projective test material 
these 


In general, 
are disappointingly small and 
much lower than one expects reliability coefficients 
to be. However, it would not be possible to say 
how much this was due to artifacts rather than 
real disagreement among the participating psychol- 
Many of the artifacts just 
lowering the validity coefficients may have lowered 
the reliability coefficients as well 


correlations 


ogists discussed as 


In light of the low interpsychologist reliability, 
it is hardly surprising that much variability was 
found in comparing the validity coefficients of the 
six psychologists who evaluated the 
The large spreads of correlations reported earlier 
imply that the differences in ability, with which 
the various participants were able to operate, were 
great, particularly when considered in the context 
of the upper and lower limits, above and below 
which the correlations would not be expected to 
extend (control correlations and 
ability, respectively). 


Same case 


indices of reli- 
These findings are in keep- 
ing with the results of the other two studies using 
Q sorts (Fisher, 1952; Little & Shneidman, 1955) 
and three other studies that utilized other methods 
of appraisal (Chamber & Hamlin, 1957; Samuels, 
1952; Symonds, 1955), all of which reported 
great individual variation in the effectiveness with 
which different psychologists were able to make 
evaluations. It seems justified to conclude from 
these various results that a projective test evalua 
tion is far from a mechanical act which proceeds 
along the lines of an X-ray reading in which any 
number of technicians, with a certain minimum 
amount of training, will arrive at the same results 
Instead, drawing inferences from projective tech 
niques would appear to be largely an art 
the skill of the individual clinician 
Thus, the question often asked “How great is the 
validity of 
only be 


whicl 


depends on 


projective technique appraisals?” can 


answered if it is 
making the appraisal 


first specified 
In regard to the experience variable, there were 
no differences among the three experience 


groups 


that were significant, and many of the small differ 
ences that were obtained went in the unpredicted 
direction 


alluded to 


The various artifacts that have already 


heen as probable causes of spuriot 


low validity coefficients would not be pertinent in 


this regard since there is no reason to suspect that 
they operated systematically, affecting one experi- 
ence group more than the others. One possible 
exception to this might be in the shortcomings of 
the criterion measure. It could be argued that the 
appraisals of the psychiatrists were so inadequate 
that their evaluations included information 

the patients that was relatively obvious and 
required relatively little skill in discerning 
projective test material. The more subtle 
aspects of the patients, which only the highly ex- 


perienced 
thi 


+ 


only 


trom the 


psychologists could discern, according to 
were not reflected in the criterion 
However, this argument 
cogency when the interpsychologist reliability co- 
It has already been noted that 
e most experienced psychologists did not agree 
any more among themselves than did the least ex- 


perienced psychologists 


S argument, 
eval lation loses its 


elhicients are noted 


Moreover, the most ex- 
perienced psychologists agreed as much with the 

experienced they agreed 
The absence of differences here 
inot be blamed on any deficiencies in the criterion 


sure 


psychologists as 


each other 


\ second way in which these findings could be 
challenged is in terms of the evaluative task itself. 
It could be argued that the kind of evaluation re- 
quired was so crude that differences in the effec- 
tiveness with which different psychologists oper- 

‘ould not be high-lighted 
argument, too, would be 


However, this 
refuted in light of two 
One is that the range of correlations for 
those clinicians evaluating the same case has, for 
all areas, been judged to be very large. Secondly, 
significant differences between groups for both 
validity and interpsychologist reliability coefficients 
have been found to exist along the personal anal- 
ysis dimension. Thus, the evaluative task in the 
is one in which differences between 
psychologists can come to light, if such differences 
exist 


findings 


study 


current 


When all these findings are considered, the lack 
iny evidence even suggesting that experience 
level plays a role in determining the effectiveness 
with which clinical psychologists evaluate projec- 
tive material, seems to me to be of major import. 
For a tar 
tl 


as I could discern from published re- 
us is the first projective technique valida- 
tion study that has systematically investigated dif- 
ilong the experience dimension where the 


Ssearc! 


ferences 
hologists were required to appraise aspects of 
that they appraise in their evervday 
The failure of other researchers 
well be 
for granted the idea that de- 
That 
e case with at least some investigators 
1 by the fact that, in studies 
Little & Shneidman, 1955), highly 
logists were deliberately 


important question may 
experience is of crucial importance 


certain 


evaluators a 


assumption that in 


was the underlying this way, 


positive results were more apt to be obtained. 
While such an assumption is consistent with what 
logic would lead us to expect, the findings of this 
study tell an entirely different story. 

The negative findings for differences along the 
experience dimension reported above are subject to 
two major qualifications. First, it is most impor- 
tant to bear in mind that the term “effectiveness” 
in this study refers only to the accuracy with 
which the clinical psychologist can make interpre- 
tations based on the projective data. This is just 
one way in which the competence of the projective 
tester is judged in actual clinical practice. Other 
skills such as organizing ideas, integrating various 
personality facets with each other, refining inter- 
pretations (that is, shading general concepts to ht 
a particular case), expressing oneself fluently, 
clearly, and with style, are also required, and these 
skills differentiate the usual report writing task 
from the Q-sorting task. These may be shown by 
future research to depend heavily on the experi- 
ence level of the psychologist. 

The fact that the participants in Group | were 
not only the most experienced psychologists, but 
were also persons with reputations tor high com 
petence in the field of projective testing, yet as a 
group, did not perform significantly more effec- 
tively than Groups IT and III, also becomes under- 
standable in light of the qualification as to what 
the term “effectiveness” applies to in the study 
For the reputations of these individuals 
earned through their teaching, writing, and super- 
visory work. The skills that underlie these func- 
tions, as well as the report writing skills mentioned 
above, can all be present to a much greater degree 
in these psychologists than in the others, yet not 
the accuracy with which they make interpretations 


were 


The second qualification to the findings reported 
above is that their applicability is limited to the 
various experience characterizing the psy- 
chologists who participated in this study. This 
qualification could be of crucial importance since 
the least experienced of the three groups could 
hardly be said to be composed of complete novices 
This group included psychologists who had_be- 
tween one and three years of clinical experience, 
where this experience was preceded by or accom- 
panied by an average of 4.7 lecture, practicum, and 
supervisory projective testing. Thus, 
most of the members of Group ITI had a moderate 
amount of contact with projective techniques. It 
may well be that the kind of ability required in 
the evaluative task in this study required, as a 
minimum for its effective execution, just such a 
moderate amount of contact with projective tech 
niques and that beyond this minimum, further ex- 
perience was of no assistance, with the relative 
degree of effectiveness with which different psy 
chologists operated, depending on other factors 
If this is true, then we would expect real beginners 
—that is, individuals who have just completed one 
or two courses in projective testing or who have 
started on, or even just completed an internship 


levels 


courses in 
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to operate significantly less effectively than the 
psychologists in Group III who participated in this 
study. This hypothesis would be simple enough to 
test in future research using the same Q sorts and 
the same criterion measures that were employed 
here 

The results reported in Table 3 indicated that 
the hypothesis positing validity for the 
evaluations of the psychoanalyzed 
was supported to some extent 
the orientation of 


greater 
psychologists 
The possibility that 
the evaluator could have influ- 
enced his ratings in two areas (Motivating Needs 
and Affects, and Infancy and Childhood Percep- 
tions), has Could the 

over-all validity coefficients of the psycho- 
analyzed psycl 


already been discussed 
higher 
logists, therefore, 


reflect, in part, 
their orientations 
psychiatrists? This 
answered in the negative. The 
orientation of all the psychoanalyzed psychologists 
was Freudian 


the greater similarity 


and the 


between 
orientation of the 
question can be 
This was only true of four non- 
psychoanalyzed psychologists. Six others described 
their orientation as that of the 
The 


as being eclectic 


interpersonal 
their orientation 
from both these 
e orientation of eight 
ion-Freudian, with at 
t influenced by the 
thinking of the interpersonal school, to the extent 
that orientation played a part in determining re- 
sults, the psychoanalyzed psychologists were at a 
Thus the fact that the over-all dif- 
was significant, 
an under- 
of the greater effectiveness of the psycho- 
analyzed psve hologists 
The 


into 


school other ten described 

, drawing heavily 

frames of reference. Since tl 


of the ten psychiatrists was 


least four and probably all 


disadvantage 
ference between the two groups 
suggests that this difference 


represents 
estimate 


differences between the two groups came 
certain aspects of 
interpsychologist reliability were considered. These 


involved 


even sharper focus when 
a comparison of the degree of agreement 
among the psychoanalyzed psychologists doing the 
wit! the degree of 
the nonpsychoanalyzed 


Same case agreement among 
doing the 
the difference be- 


coethcients of 


psve ologists 
over-all scores 
reliability 


Same case For 
tween the mean the two 


groups was .13 compared to a difference of .07 for 


their mean validity coefficients. The former dif- 
ference is significant at the 1% level, while the 
latter is significant at the 5% level. For individual 


areas, the differences for reliability were also much 
greater than the differences between the validity 
coefficients in the following three areas 

Defenses validity coefficients .04 
(insignificant )—differences in reliability coefficients 
14 (significant at the 5% level) 


Differences in 


Character Traits. Difference in validity coeff- 
cients (insignificant)—difference in reliability 
coefficients .29 (significant at the 1% level) 


Infancy and Childhood 


in. validity 


Perceptions. Difference 
(significant only if the 
10 level) — 
16 (significant 


coefficients .11 
significance criterion 1s relaxed to the 
difference in reliability coefficients 


at the 5% level) 
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Two explanations will be posited which I believe 
are most apt to account for the fact that the dif- 
ferences between the psychoanalyzed and non- 
psychoanalyzed groups were greater for reliability 
than for validity. One of these revolves around 
the inadequacies of the criterion measure. It may 
be that these inadequacies minimized the actual 
degree of difference between the two groups as 
measured by their validity coefficients. The differ- 
ences in reliability coefficients, on the other hand, 
not being dependent on the criterion measure, 
would then reflected more just 
how much more effective was the psychoanalyzed 
group 


have accurately 


The second explanation would revolve around 
the relationship between orientation and the per- 
sonal dimension. Since orientation could 
have affected ratings in some areas, two psychol 


analysis 


ogists evaluating the same case who were of the 
same orientation could be expected to agree more 
with each other than two psychologists of different 
orientations. Inasmuch as all the psychoanalyzed 
psychalogists were of orientation while the 
nonpsychoanalyzed psychologists were hetero- 
geneous in this regard, this is what might under- 
lie much of the greater reliability in the former 
group 


one 


In the area of Infancy and Childhood Percep 
tions, the latter explanation could be at least par 
tially applicable. For, as was elaborated upon 
earlier, the degree of emphasis given to certain 
items in this area could well be a consequence of 
theoretical orientation. To the extent that this was 
the case, the higher reliability figures would reflect 
an artifact and not a more accurate index of dif 
ference between the psychoanalyzed and nonpsy- 
choanalyzed groups 

In the area of Defenses, however, the explana- 
tion in terms of an artifact would have much less 
weight for there is no reason that a 
particular orientation would psychologist 
to weight one type of defense more heavily than 
There are differences between the two 
psychiatric schools of thought in their understand- 
ing of what the various defenses owe their origin 
to, but members of both groups would agree that 
these various defense mechanisms do exist. It is 
true that most of these concepts and the terms in 
which the 


to believe 
lead a 


another 


concepts are expressed originated in 
Freudian psychoanalytic theory, so that psychol 
a Freudian orientation might have felt 


particularly comfortable dealing with them 


ogists with 
How 
ever, these concepts and terms play a considerable 
part in the members of the 
Thus, the 


thinking of inter 


fact that the 
coethcients between the 
than the 
coefficients, more likely 
shortcomings of 


personal school as well 


difference in validity two 
difference 
reflects the 


the criterion measure. The lat 


was quite a bit lower 


groups 


in reliability 


ditference, therefore, would more accurately reflect 


the degree of greater effectiveness of the 


psvcho 


analyzed group 


In the 
less re 


area of Character Traits, there is even 
ason to believe that theoretical orientation 
psychologists influenced their evaluations. 

None of the traits mentioned has priority over 
others in the theory of either school. 
the actual terms utilized were employed in the 
long before the advent of psychoanalysis. 

hus, for this area, the degree of difference be- 

tween reliability 


of the 


Moreover, 


coefficients is probably a 
estimate of the 
sychoanalyzed group 
When 
and 


more 
greater effectiveness of 
lifferences between the psychoanalyzed 
nonpsychoanalyzed groups for both validity 
interpsychologist reliability are considered, the 

group's effectiveness can be taken as nota- 
than the latter’s. To what is this 
greater effectiveness due? It has been my assump- 


tion ft 


um 
reater 


at the answer to this question lies in certain 
benefits that accrue from undergoing psychoanal- 
ysis. It is possible, however, that this assumption 
is unjustified. Could it not be that other 
ctor led both to these psychologists’ being more 
in their work with projective techniques 
their entering this type of treatment? 
Would they then not have attained just as high 
correlations if they had not entered psychoanalysis ? 
I believe that the first explanation which posited a 
and effect relationship is the more likely 
of two kinds of The first is 
and has already presented as the 
rationale underlying Hypothesis III, in which were 


some 


proficient 


and to 


auUuse 
ev idence 
been 


given the reasons why undergoing such a form of 
treatment should increase the effectiveness with 
hich psychologists interpret test data.* The 
md is subjective, consisting of comments made 

by some of the psychoanalyzed psychologists to the 
effect that they believed that their effectiveness 
with projective tests increased as a result of psy- 
lysis. However, neither of these reasons can 


choan 


be accepted as proof, for it is not unusual for logic 


to bow to empiricism, and moreover, it is possible 


that just 


as logical a rationale could be presented 
to support the alternative explanation. As for the 
hologists’ comments, such self-evaluations can- 
always be trusted. It would be crucial in 
this issue to compare the effectiveness of 

1 group of psychologists just 


analysis with the 


beginning 
psychoanalyzed 


pss cho- 
group in this 


study, using the same QO sorts and the same crite- 


measure. Only three of the twenty non- 
inalyzed psychologists had just begun psy- 


lysis so that no such 


comparisons could be 


iddition to the given in the 

ion, general therapeutic gains 
hoanalysis should be mentioned. To the 
hoanalysis succeeds, anxiety and 


it a psvel 
s should be minimized and more conflict- 


explanation 


eses section, the 


at the analysand’s disposal 


lead to more effective func- 


area as well as others 


TIO 1 
psvel 
In 
Hy potl 
ot ps\ 
extent 
inhibitior 
These changes would 
tioning in the work 


made here. Until such a step is taken, my explana- 
tion for the results rests on somewhat shaky 
ground. 

Can only psychoanalyzed psychologists evaluate 
projective test material with a relatively high de- 
gree of effectiveness? An examination of the dis- 
tribution of both validity and interpsychologist re- 
liability scores among both the psychoanalyzed and 
nonpsychoanalyzed groups indicates that this is not 
the regard to validity coefficients for 
over-all agreement, 37% of the nonpsychoanalyzed 
group achieved coefficients that were at least as 
high as the average coefficient for the psycho- 
analyzed group. Similarly, for over-all inter- 
psychologist reliability, the coefficients for 33% of 
the nonpsychoanalyzed pairs were at least as high 
as the average reliability coefficient for the psycho- 
analyzed pairs. Thus, one can assume that some 
psychologists, either without treatment or as the 
result of successful psychotherapy, are in sufficient 
touch with intrapsychic forces that are usually un 
conscious and are sufficiently free of debilitating 
anxiety and conflict so that they function with a 
relatively high degree of effectiveness in evaluating 
projective test material 


case. In 


Regarding the converse, there is the question of 
whether any psychoanalyzed psychologists were, 
in a relative sense, ineffective in their evaluations 
The answer would be in the negative. In terms of 
over-all agreement with the criterion, the lowest 
validity coefficient of any psychoanalyzed psychol- 
ogist was .25, whicli is only slightly lower than the 
mean validity coefficient for all 30 psychologists of 
.28. Similarly, the lowest reliability coefficient for 
interpsychologist agreement among the  psycho- 
analyzed group is .29, not much lower than the .33 
whiich represents mean reliability for interpsychol- 
ogist agreement when all pairs of psychologists in 
the same personal analysis group are considered 
Thus, if one generalize from the present 
sample, it would seem that the fact that a psychol 
been psychoanalyzed offers some assur 
his projective test evaluations will be, 
least accurate 


can 


ogist has 
ance that 
in a relative sense, at reasonably 

The one finding that was diametrically opposite 
to what was hypothesized also deserves comment 
In the area of Interpersonal Behavior, for the 
psychologists as a total group there was a decreas- 
ing degree of agreement between their evaluations 


I be- 


an artifact 


and the psychiatrists’ successive evaluations 
lieve that the reason for this involved 
It seems highly probable that the psychiatrists were 
not able to adhere to the instructions given them 
during their second and third evaluations, that they 
evaluate the patient in terms of how he was when 
he first entered treatment, excluding chances that 
took place during the period of therapy. Such in- 
structions would be most difficult to For 
there was no way in which a psychiatrist could be 


follow 


certain that the behavior the patient exhibited, or 
described after being treated for some time, was 
truly a new mode of interaction 
simply have suppressed such action with the psy 


The patient may 
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chiatrist and even suppressed describing the be- 
havior until he felt more at home in the therapy 
situation, Overt behavior might well change, par- 
ticularly during late adolescence and young adult- 
hood. Thus, the psychiatrist may have rated items 
during the second and third evaluations, thinking 
that he was describing the patient as he originally 
was but in a truer light, while in reality he was 
describing new kinds of behavior 

Why, then, did not the same error occur in the 
other five areas that the psychiatrists evaluated ? 
The answer would lie in the nature of the person- 
ality variables tapped in the other Cer- 
tainly Infancy and Childhood Perceptions of 
parental figures which, by definition, refers to the 
patient's could not For 
and Symptoms, no change would be anticipated 
since the nosological entities were stated in terms 
of vulnerability. Thus, even if a patient lost a 
particular symptom during such brief treatment, 
he would still be vulnerable to its recurrence in 
the future. Character Traits referred to lifelong 
reaction patterns, Motivating Needs and Affects to 
drives and and Defenses to habitual 
to anxiety Neither life nor 
therapy of such short duration could be expected 


areas 


past, change Diagnosis 


basic tears, 


responses events 
to bring about alterations in these three areas, even 
though behavioral changes might occur 

hypothesized differences 
along the experience dimension for 
ment with the psychiatrists’ successive evaluations, 
parallels the negative findings for comparisons of 
the three groups for their validity and inter- 
psychologist reliability Thus, here is 
another skill of the clinical psychologist, for which 
evidence is experience 


The failure to find the 
increased agree- 


coethcients 
lacking, that degree of 
affects performance 

In terms of differences along the personal anal- 
Defenses the find- 
ings paralleled the results of comparisons between 


vsis dimension, for the area of 


the two 
reliability 


tor validity and interpsychologist 
this instance, in a 
Thus, another advantage that ap- 
pears to accrue trom a clinical psychologist under- 


going psycl 


groups 

coethcients, only in 
positive sense 
ioanalysis, subject to the reservation re- 
garding cause and effect relationships made earlier, 
is that it ability to detect central and 
hidden defensive patterns that are obscured from 
the patient has 

Perhaps the 
what 
is chaff and what is in himself allows him 
to make the same differentiation in patients whose 
protective test material he evaluates 

In light of the consistent differences found be- 
tween the psychoanalyzed and nonpsychoanalyzed 
future research on the 
differential ability with which clinical psychologists 


evaluate projective test data would do well to in- 


increases his 
detection by psychotherapists until 


been in treatment for time 


ral 
psycnoanalyzed 


some 
psychologist’s awareness of 
wheat 


psychologists in this study, 


vestigate this variable. The fact that the group 
designated as psychoanalvzed in this study included 
only those psy hol ists who had undergone 
Freudian psychoanalysis should be kept in mind, 
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however, since it is my belief that the combining 
of psychologists who receive various types of in- 
tensive or dynamic therapies into one group would 
obscure differences that actually exist. 


SUMMARY 


This study was undertaken to investigate 
the validity of projective technique evalua- 
tions. The patients whose test material was 
to be appraised were 10 young adult males 
who entered psychotherapy at the Psychi- 
atric Clinic attached to the Court of Special 
Sessions of the City of New York. Each 
had administered to him a_ Rorschach, 
Thematic Apperception Test, House-Tree- 
Person Drawings, and the Most Unpleasant 
Concept Test before entering treatment. 

The test material was evaluated by a 
group of 30 clinical psychologists. These 
psychologists were divided into three sub- 
groups depending on degree of professional 
experience and two subgroups depending on 
whether or not they had undergone Freud- 
ian psychoanalysis. Each psychologist eval- 
uated the protocols for two patients and 
each patient was evaluated by six psychol- 
ogists. 

The degree of validity of the psychol- 
ogists’ evaluations was determined by cor- 
relating them with evaluations made by the 
psychiatrists treating the patients after 35 
or more therapy sessions. The psychiatrists 
were also asked to evaluate the patient at 
two earlier intervals during treatment to 
determine if there was increasing agreement 
between their evaluations and the psychol- 
ogists’ evaluations as the psychiatrists be- 
came better acquainted with the patients. 

The method of evaluation for both psy- 
chologists and psychiatrists involved the use 
of Q sorts. There were six O sorts, one for 
each of six personality areas to be ap- 
praised. These areas were: (a) Defenses: 
(b) Motivating Needs and Affects: | 
Character Traits; (d@) Diagnosis and Symp- 
toms; (¢) Interpersonal Behavior; (f) In- 
fancy and Childhood Perceptions of Paren- 
tal Figures. 


( 


A summary of the results follows: 


1. The psychologists as a total group were 
able to evaluate the projective test material 
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to a degree significantly greater than 
chance, both over all and for five of the six 
areas considered separately. For the sixth 
area (Defenses) there was a tendency in 
this direction, 

2. There were no significant differences 
between the three experience subgroups in 
the size of their validity coefficients. There 
were also no significant differences in inter- 
psychologist reliability in terms of experi- 
ence level. 

3. Those psychologists who had under- 
gone psychoanalysis had signifi- 
cantly higher over-all validity coefficients 
than those psychologists who had not re- 
ceived this form of treatment. While dif- 
ferences between these two subgroups were 
not significant for any of the six areas con- 
sidered separately, in three of the areas 
there were tendencies in the same direction. 
There were also significant differences in 
interpsychologist reliability, both over all 
and for three of the areas considered sepa- 
rately. In all instances there was greater 
agreement among those psychologists who 
had undergone Freudian psychoanalysis 
than among those who had not. 

4. The psychologists as a total group 
agreed to a significantly greater degree with 
the psychiatrists’ later evaluations than with 
their earlier ones in the area of Character 
Traits. In the area of Motivating Needs 
and Affects there was a tendency in the 
same direction. In terms of over-all scores, 
and for the other four areas considered 
separately, there was no increasing agree- 
ment. In one of these areas (Interpersonal 
sehavior) the psychologists showed signifi- 
cantly decreasing agreement with the psy- 
chiatrists’ successive evaluations. 

5. In the experience vari- 
able, there were no significant differences 
between the three subgroups in regard to 


regard to 


their ability to show increasing agreement 
with the psychiatrists’ successive evalua- 
tions. 

6. In regard to the personal analysis vari- 
able, those psychologists who had under- 
gone Freudian psychoanalysis showed sig- 
nificantly greater increasing agreement with 
the psychiatrists’ successive evaluations in 
the area of Defenses than those psychol- 


3 
4 
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ogists who had not received this form of 
treatment. In the area of Infancy and 
Childhood Perceptions, there was a tend- 
ency in the same direction. For over-all 
scores, and for the other four areas consid- 
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APPENDIX 


Q SORTS 


DEFENSES 30. The patient often uses intellectualization as a 


means of coping with anxiety arousing hostile 
1. The patient heavily relies on repression. needs 
2. Intellectualization is prominently used as a 
defense against sexual needs. = 
3. Undoing is a frequently used defense mechan- Il. MortivatinG Neeps anp AFFECTS 


ism. 
4. Isolation of feeling appears to be a major 
defense. 


1. A major source of anxiety for him (her) is 
tear of separation from maternal figures. 


ae. : . 2 he patient suffers much guilt over hetero- 
5. The patient puts great reliance on the defense : 
sexual impulses. 
of avoidance. 4 tri th 
ompetitive Strivings with 1e parent ot his 
6. The patient frequently projects his inade- I 
(her) own sex play an important motivating 
quacies onto others. rol 
7. The patient often alleviates anxiety through 
: 4. The fear of losing control over aggressive 
aggressive acting out t t fi td 
, ; : impulses 1s present to a significant degree 
Overcompensation iS a major defense mechan- I 
rt 5. The patient has prominent fears of being 


9. The patient often uses depersonalization for destroyed 


defensive purposes 6. A marked need to replace the same sex parent 
10. Displacement is one of the most heavily relied _ = evident 
upon defenses. 7. Fear of helplessness is prominent. { 
11. The patient often uses the defense of reaction 8. Anal sadistic strivings play an important role i 
he > > 
formation against unacceptable passive long in the patient's illness. 
ings. 9%, Incestuous strivings toward the opposite sex 
12. There is a frequent retreat into fantasy life parent 1s a strong motivator. 
13. The patient has a marked tendency to with- 10. The search for an omnipotent father hgure 


draw from environmental stimulation. an important role in motivating the 


14. The patient frequently feels gay and/or friv- 


olous as a defense against depression 11 ed to maintain infantile omnipotence is 
15. The patient often identifies with the aggressor n important motivator 
as a means of warding off anxiety. 12. The fear of losing control over heterosexual 
16. Somatization is frequently used in an attempt impulses is present to a significant degree. 
to bind anxiety 13. The fear that his (her) love offerings will be 
17. The patient uses reaction formation a good rejected by maternal figures is prominent. 
deal as a defense against aggressive impulses 14. Homosexual needs play a prominent motivat- 
18. The defense of intellectualization as a means ing role 
of coping with unacceptable dependency needs 15. The patient suffers much guilt over hostile 
is prominent impulses 
19. The patient resorts to much sexual acting out lo. A search for maternal love is an important 
as a way of alleviating anxiety. motivator tor the patient 
20. The patient often reacts in a counter phobic 17. The patient shows prominent exhibitionistic- 
fashion when in a threatening situation voyeuristic needs 


21. Suppression of impulses frequently takes place 18. A sense of sexual inadequacy frequently moti- 
22. The patient frequently hates those of the vates the patient. 
opposite sex as a defense against his fear of 19. The need to assert his (her) independence is 
them al portant motivating tactor. 
23. The patient often uses depression as a defense 0. ‘The wish for punishment motivates much of 
against aggressive feeling the patient’s behavior 
24. The patient makes frequent use of sublimation 21. Feelings of inferiority are a strong motivator. 
L 25. The patient often turns aggressive feeling in >? There is a strong fear of castration 
on the self 23. Fear of separation from paternal figures is an 
26. The patient frequently projects his hostile mportant motivator 
impulses 24. An excessive need for protection and security 
27. Denial is used as a major defense mechanism s present 
f 28. The patient frequently utilizes regression as ; 25. The fear that his love offerings will be rejected 
defense maneuver by paternal figures is strong 
29. The patient often projects sexual feeling by 26. The patient is strongly motivated by feelings 
seeing others as desirous of him (her) of shame over unacceptable impulses 
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VALIDITY OF EVALUATIONS MADE 


The patient is much motivated by the need of 
approval from maternal figures 

Strong oral-dependency needs are an important 
motivator. 

The patient is much motivated by the need 
for approval from paternal figures 

Feelings of rivalry with siblings play an im- 
portant role in the patient's illness 


IIT. CHARACTER TRAITS 


This is a very ambitious individual. 
Orderliness is a prominent character trait 

He (she) is characterized by much impul- 
Sivity 

He (she) would be considered a highly self- 
sufficient person. 

The patient is a very rigid individual 

The patient is much concerned with human- 
itarian causes 

He (she) is notably self-righteous 

Evasiveness is an obvious characteristic of this 
patient. 

The patient frequently identifies with authority 
figures 

The patient is often not able to complete tasks 
he begins. 

This individual is much concerned with phys- 
ical strength. 

The patient is highly suspicious. 

Grandiosity is a noteworthy character trait 
This patient has many masculine character- 
istics (in regard to manner or interests). 
Ambivalence is a noteworthy character trait 
This is a highly egocentric individual 

The patient frequently identifies with the 
underdog 

This individual is noticeably feminine in 
gard to his (her) manner or interests 

This is a highly optimistic individual 

The patient is very class conscious 

The patient is a highly creative person 

This is a very moral and/or ethical individual 
The patient is characterized by his (her) great 
concern with wealth and/or power 

He (she) is very naive 

This is a very parsimonious individual 
Lability is an outstanding character trait 

He (she) is a very obstinate person 
Perseverance characterizes this person to a 
large degree 

The patient is notably haphazard in his (her) 
upproach to things 

Much indecision characteristic 

patient 


IV. DiaGNosis AND SyMPTOMS 


The patient is vulnerable to hysterical phobias 
The patient is a passive-aggressive character 
The patient is vulnerable to specific obsessions 
The patient shows noticeable mood swings be- 


ween elation and depression 


FROM PROJECTIVE TECHNIQUES 


The patient suffers from an obsessive-compul- 
sive character disorder 

The patient is vulnerable to anxiety states. 
The patient is vulnerable to psychotic depres- 
sion 

The patient is vulnerable to psychosomatic 
symptoms. 

The patient is vulnerable to simple schizo- 
phrenia. 

The patient suffers from reactive depressions, 
frequently 

The patient is vulnerable to paranoid schizo- 
phrenia 

The patient is vulnerable to sexual perversion. 
The patient is suffering from an_ hysterical 
character disorder. 

The patient shows noteworthy psychopathic 
features 

The patient is vulnerable to sexual impotency 
( frigidity). 

The patient is suffering from a_ narcissistic 
character disorder 

The patient is a schizoid character. 

The patient is vulnerable to hysterical con- 
version symptoms 

The patient is vulnerable to catatonic schizo- 
phrenia 

The patient 1s a paranoid character 

The patient is vulnerable to compulsive rituals. 


patient is a productive or genital char- 


patient is a passive-dependent character. 
“he patient suffers from a_ borderline state 
utilizing both psychotic and neurotic mechan- 
isms 
Organic brain damage is present 
The patient is vulnerable to manic states 
The patient is vulnerable to iz or alcohol 
addiction 
The patient is vulnerable to a neurotic depres- 
The patient is v able to amnesia, multiple 
personality, or other dissociated conditions, 
The patient is vulnerable to 


schizophrenia 


hebephrenic 


V. INTERPERSONAL BEHAVIOR 


The patient devaluates and mocks those of the 
opposite sex 

The patient is compliant with authority fig- 
ures 

The patient relates to others in an overly intel- 
lectual manner 


The patient sexualizes his (her) relationships 
most persons of the op 
patient act a domineering manner to- 
rd those of the ime sex 


posite sex 


“he patient is inclined to play the role of a 
“clown” when with others 
vatient has a Pollyannaish orientation in 


iting to others 
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ships with those of the opposite sex. 

The patient withdraws from close contact with 
those of his (her) sex. 

The patient generally assumes a 
compliant role when with peers 

The patient acts in a 
when with peers. 


passive- 


self-assertive manner 
The patient acts toward others in a subtly 
negativistic fashion. 

The patient is timid in relating socially to 
those of the opposite sex. 

The patient 
from others. 
The patient withdraws from 
with those of the opposite sex. 


remains inaccessible and aloof 


sexual contact 


The patient acts demandingly toward those of 
the opposite sex. 

The patient play-acts much of the time. 

The patient tends to control those of the op- 
posite sex through his (her) weakness. 

The patient acts in an aggressively grasping 
attitude toward others. 

The patient 
others 


plays a self-effacing role with 


The patient acts competitively toward peers. 
The patient acts defiantly with authority fig- 
ures 

The patient is pretentious and/or exhibition- 
istic in relating to others. 

The patient dependently clings to those of the 
opposite sex 

The patient goes to great lengths to impress 
others with his (her) juvenile innocence and 
harmlessness 

The patient tends to control those of the same 
sex through his (her) weakness. 


The patient tries to domineer those of the 
opposite sex 
The patient acts in an ingratiating manner 


toward authority figures. 
The patient is timid in 
those of the same sex. 


relating socially to 
The patient acts competitively toward author- 
ity figures 

VI. INrFancy AND CHILDHOOD as 
PERCEIVED BY THE PATIENT 


The patient felt deprived in his (her) quest 
for receptive-dependent gratifications 

The patient felt that the father was disap- 
pointed in him (her) frequently. 

The patient felt unwanted by the father 


The patient is sadistic in his (her) relation- 
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The patient saw the parent of the same sex 
as too powerful to be opposed. 
The patient felt that it was dangerous to be 
assertive except in subterfuge. 

The patient saw the mother in the role of 
protector against the patient’s father. 
The patient felt that the father was 
demanding of him (her). 
The patient pe received the 
and submissive 

The 
hetic 


over- 


father as passive 


father was seen as crude and unsympa- 


The patient felt the mother showed love only 
when her high standards were met. 

The patient felt the father showed much love 
for him (her). 

The patient saw the father as a bully 

The patient felt that infantile behavior would 
be severely punished by one or both parents. 
The mother was seen as too weak 
help to the patient. 

The patient felt it threatening to 
identify th the opposite sex than the same 


to be of 


was less 
parent 

The patient felt that the mother was overly 
punitive 

The patient saw the father in the role of pro- 
tector against the patient’s mother. 

The patient felt that the same sex parent was 
not enough to identify with. 

The patient felt that his (her) mother was 
disappointed in him (her) frequently 

The mother was perceived as understanding 
and/or sympathetic. 

The patient felt that his (her) mother did the 
thinking for him (her) allowing little auton- 
omy of action. 


strong 


The patient felt that any hostile action or ex- 
pression would be quickly squelched by one 
or both parents 

The father was perceived as patient and under- 
iding 

patient felt 


im (her). 


that the mother was aloof 
was perceived as overcontrolling. 

The mother was felt by the patient to be 

untrustworthy 

The patient felt that one or both parents was 

overpermissive, letting him (her) “get 

with murder.” 

The patient 


and 


away 


perceived the father as unreliable 
undependable 

The patient felt that the opposite sex parent 
was sexually seductive toward him (her) 

The patient perceived the mother as “s! 
ing” love and affection on him (her) 


ower- 
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